Simple explanation on some basic FX shader programming

[wonderfulmore]

How it started:

Tweaking is so time consuming, I am considering quitting. But I am still loving shader programming although I know very little. I have done some research and built my effects shader from ground up. I think I might be able to contribute to this awesome community by explaining some ideas and programming on pixel FX shader, starting from tone mapping and adaptation, to color/luminance calculation, then "normal" bloom, curved bloom, HD6's crisp bloom, also highlight/shadow enhancements etc. Is anyone interested? Anyway you can find all my codes in Imperfect ENB. Just ignore me if I am overthinking or over valuing my stuff

Ok, so I guess I'll start straight away. I'll try to be as concise as possible. It's my personal understanding, hopefully they are all correct or at least not misleading.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1. Introduction to enbeffect.fx
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

ENB replace the default game engine's rendering. It does many computation and at some stage calls the .FX files and use the shader techniques defined there. Think the FX files as customizable slots that Boris has left for us to develop some plugins.

I'll concentrate on enbeffect.fx, simply because I know more about this one.

enbeffect.fx is where the almost last stage of post processing happens. Before it, all the complex effects are calculated, bloom, lenz, rays are sampled. It's more like a final assembling and tweaking place for all post processings.

The structure is quite simple. First are the inputs, they are external parameter passed from ENB and some pre processed textures like original color, bloom, adaptation, palette etc.

float4 tempF1; //0,1,2,3
float4 tempF2; //5,6,7,8
float4 tempF3; //9,0
float4 Timer; // x=generic timer in range 0..1, period of 16777216 ms (4.6 hours), w=frame time elapsed (in seconds)
float4 ScreenSize; // x=Width, y=1/Width, z=ScreenScaleY, w=1/ScreenScaleY
float ENightDayFactor; // changes in range 0..1, 0 means that night time, 1 - day time
float EInteriorFactor; // changes 0 or 1. 0 means that exterior, 1 - interior
float EBloomAmount; // enb version of bloom applied, ignored if original post processing used

texture2D texs0;//color
texture2D texs1;//bloom skyrim
texture2D texs2;//adaptation skyrim
texture2D texs3;//bloom enb
texture2D texs4;//adaptation enb
texture2D texs7;//palette enb

Then the vertex shader function. It can manipulate image with some geometry info, rather than treat the image as a flat paint. I am not good at vertex shader, besides it's rarely used in this stage of post processing. I'll just skip it.

VS_OUTPUT_POST VS_Quad(VS_INPUT_POST IN)

Next is where most magic happens, the pixel shader process each pixel on the screen with the same code written in it, think it as a crude photoshop written in C like code.

float4 PS_D6EC7DD1(VS_OUTPUT_POST IN, float2 vPos : VPOS) : COLOR

At the end, shaders are wrapped in a technique, which will be used by ENB rendering:

technique Shader_ORIGINALPOSTPROCESS
{
pass p0
{
VertexShader = compile vs_3_0 VS_Quad();
PixelShader = compile ps_3_0 PS_D6EC7DD1();

ColorWriteEnable=ALPHA|RED|GREEN|BLUE;
ZEnable=FALSE;
ZWriteEnable=FALSE;
CullMode=NONE;
AlphaTestEnable=FALSE;
AlphaBlendEnable=FALSE;
SRGBWRITEENABLE=FALSE;
}
}

Now let's go back to pixel shader, where most things will go into.

Pixel shader is basically a pipeline, some things go in, one thing go out.

Things that go in to the shader, these are the materials at our disposal:

* Original color

_v0.xy=IN.txcoord0.xy;
r1=tex2D(_s0, _v0.xy); //color
_oC0.xyz=r1.xyz; //for future use without game color corrections
float4 color=_oC0;

Here _s0 is the texture contains original image before processing. First sample it to r1, than assign to _oC0, finally put it in to variable "float4 color". IN.txcoord0.xy is the coordinates of the pixel currently being processed. This will go through all pixels on the screen one by one, for each pixel we run this same shader once. "color" is a 4 dimension vector which contains: (x:red, y:green, z:blue, w:alpha). Usually when deal with colors, we manipulates red/green/blue, which can be represented by "color.xyz", you will see it a lot later.

Similarly, two other very important inputs are:
bloom texture, adaptation texture

bloom texture is simply a blur of original image, looks simple but very useful, with some magic it can create the awesome bloom effects.

adaptation is a small icon sized (may not be accurate about actual size, but it's small) image resembling the original image. when you set AdaptationSensitivity to 0 in enbseries.ini, it represents the average color (or luminance) or the image, when set to 1, it represents the max (not exactly but a "blurred" max) color/luminance of the image.

There's also palette texture, basically the same as the palette file you put in ENB folder.


The output of pixel shader is: a pixel. represented by "color.xyzw" (or simply "color"), it was twisted, mixed with different texture, tortured and finally returned to ENB as the product of post processing:

_oC0.w=1.0;
_oC0.xyz=color.xyz;
return _oC0;

That took longer than I thought, I'll cover HDR and tone mapping next time, which are the foundations of all effects used in enbseries.ini.

[--JawZ--]

I think it would be a great idea for people starting to get interested in ENB and want's to alter their .fx files codes slightly, but don't know where to start. So if you want, go right ahead with it
If you have the time and drive to do it of course

[Spellbound]

I think this is a brilliant idea! I, for one, am completly confused with the many types of blooms and such. I don't know if this applies, but I'm currently experimenting with Bokeh DoF in an RTS setting - so maybe you could cover this as well if you have the time

[--JawZ--]

And it might inspire other code developers to share and explain their codes and other functions as well.

[wonderfulmore]

I think this is a brilliant idea! I, for one, am completly confused with the many types of blooms and such. I don't know if this applies, but I'm currently experimenting with Bokeh DoF in an RTS setting - so maybe you could cover this as well if you have the time

sorry but sampling is an area that I always wanted to but haven't touched, will try my best on others

[wonderfulmore]

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2. Tone mapping
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++

To understand tone mapping we first need to talk about HDR (High Dynamic Range) and LDR (Low Dynamic Range).

When we output colors to the displaying device, the values of color range from (0,0,0)-pure black to (1,1,1)-pure white. this is the LDR. easy for displaying device to process, because all values fall into [0,1] range.

However LDR is a pain for lighting and effects computing. Because in real world, the lights can be tens or thousands times brighter than shadows. It's hard to calculate "realistic" lighting with in that small range.

The solution is also simple. we compute everything in a much larger (you can think it as infinite though it's not) linear range, after everything is done, compress (or you can say map) it back to [0,1]. That's HDR rendering and tone mapping.

The rest is just math. We have an input: color.xyz, of [0,maybe infinite] values (HDR), then we want to apply a formula to it, map it to [0,1] range (LDR), in the end output the LDR result to displaying device.

Off-topic: why I don't believe in SweetFX. Because without tapping into rednering core, it only fixes the final color in LDR, where "true" bloom, tone mapping, and many other great effects cannot be calculated in optimal manner. There are a lot of limitations on what it can achieve due to LDR.

Back to topic.

A most straight forward example of tone mapping formula is: y = x/(1+x) when x=0, y=0, when x=infinite, y=1. So whatever feed into the formula is mapped to [0,1]. Sounds fancy, but there's a problem. Consider when x=50, y=50/(1+50)=0.9804, when x=60, y=0.9836. In HDR, light intensity of 50 and 60 have 20% difference, which is significant. But after applying our tone mapping formula, they only have 0.0032 difference on screen, which probably means no difference to eyes--they all look nearly pure white.

Consider another scenario, which is farely normal bright, the light range [0,2] in the scene. See where it is mapped to: [0,0.6667], which means all colors in the scene is clamped in the lower 2/3 range of monitor's displaying capacity, which is too dark and unrealistic.

Above problems all come from the fact that this tone mapping is static. 0.5 is always mapped to 0.33, 2 is always mapped to 0.67, 10 is always mapped to 0.91 etc. Monitor's contrast is already poor comparing to real world lighting. What we want is to stretch the range if it's too small, revealing more details in shadows; and compress range if it's too big, so that the intense lights are not pure white. This requires the tone mapping to be dynamic and really smart.

In other words, the tone mapping has to "Adapt" to the scene. which goes into the field of Adaptation, we will come to that later, now let's focus on tone mapping formula itself.

Now we know the formula y=x/(1+x) has the disadvantage: static, fixed. To find a better way to do this, programmers created all crazy formulas. Some of them don't make sense at all upon first look but somehow work.

The one I personally prefer is a variation of y=x/(1+x):

y = x*(1+x/w^2)/(1+x)
where w is a parameter

The feature of this formula is that it maps range [0,w] to [0,1]. By changing the value of w I can exactly control how much range to be tone mapped.

For example, when w=0.5, this formula stretches [0,0.5] range to [0,1], making the intensity of 0.5 appear pure white on screen. when w=3, intensity of 3 is mapped to pure white, colors >3 will be pure white as well, colors <3 will be compressed. That's why people usually use "w" or "Lw" for this parameter, it stands for Luma White.

For you reference, variations of above two formula are usually referred as Reinhard tone mapping. Famous for Reinhard's paper on applying these formulas in tone mapping. You can find that original paper by googling if you are interested.

The tone mapping formulas in default enbeffect.fx are also similar to Reinhard's, but Boris created them with much more parameters and modifications. The purpose is all the same: mapping a given HDR range to [0,1] for display.

Now that we have control over the tone mapping range (white point), there's a new problem: how to determine the white point for each scene?

As mentioned before, over compressing makes the image dark/damp, not enough compressing makes the brightest lights all white, losing highlight details. This requires the shader to analyze each scene and intelligently determines the range. As said, this is Adaptation. Similar to Eye Adaptation, when human eyes adapts to different light conditions, they basically do a biological tone mapping: adjust the scene to a range that can be best perceived by retina (my understanding )

I'll cover adaptation next time, the concept is simple but difficult to implement/tweak. Also i'd like to cover other aspects of tone mapping, like some may notice ENB tone mapping changes saturatino/hue, I'll explain the cause and come to the topic of color manipulation.

++++++++++++++++++++++++++++++++++++++

A few sample codes for tone mapping:

ENB PostProcess 1-3

Code: Select all

color.xyz=(color.xyz * (1.0 + color.xyz/lumamax))/(color.xyz + EToneMappingCurveV1);

ENB Postprocess 4

Code: Select all

color.xyz=color.xyz/(color.xyz+EBrightnessToneMappingCurveV4);

My implementation of Reihard's tone mapping

Code: Select all

float lumaNew = lumaIn*(1+lumaIn/pow(lumaWhite,2))/(1+lumaIn);

[Marty McFly]

May I use parts of the first post for my own shader explanations (wanna create guide to effect.txts for the gta sa guys since it seems I'm the one of the only code monkeys on the gta sa scene)?

[wonderfulmore]

May I use parts of the first post for my own shader explanations (wanna create guide to effect.txts for the gta sa guys since it seems I'm the one of the only code monkeys on the gta sa scene)?

sure

[wonderfulmore]

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3. Adaptation: Tone mapping
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Well I am not good at writing in English nor do I have much spare time for this. But I don't feel like leaving things half way. So I'll try my best to finish all that I have to say.

Adaptation adapts everything: color, dynamic range, brightness, gamma etc. in certain patterns so that the HDR image can fit into the LDR screen display and give a attrative look to human eyes.

The basic adaptation is for dynamic range, or you can say tone mapping. Because other adaptations are mainly for fine tuning, but HDR image simply can't be displayed without proper tone mapping with proper adaptation.

So I will focus on tone mapping adaptation first.

Like I said last time, tone mapping is all about compress an HDR to LDR. But the optimal range changes per scene. There are many ways to develop adaptation, but I can only give the example of how I developed it in ENB context.

The basic idea is to guess the optimal range by the average brightness (or call it luminance) of the scene. For instance:

I first go to the darkest scene where I'd like to adapt, where the average luminance is 0.2. And I adjust tone mapping range until I find [0,0.4] is the proper range that makes the image look good.

Then I go to the brightest scene in the game, where the average luminance is 2, I experiment around and decide [0,4] to be the ideal range.

Now there's a relation between average luminance and max tone mapping range:
When Luminance=0.2, Max Range = 0.4;
When luminance=2, max range = 4;

Does that mean when luminance=1, max range of 2 would be a good looking value?
My experience is no. Most of the time, tone mapping adaptation doesn't come linear. The ideal Luminance-Range relation is usually a curve: range increase slow at first, and more rapidly towards the brightest end, or the opposite way, all depends on your lighting setup.

To achieve that, I use a power function to adjust the rate of Range increase as Luminance grows. So I go to a scene where the luminance is in between, for example luminance = 1, then adjust the power number until I get a good looking image.

If you are familiar with lerp() function, above behaviour is like a power curved lerp. If you wonder how it works programmingly, here's the piece of code:

Code: Select all

void getCurve(float inVal, float inMin, float inMax, float outMin, float outMax, float curve, inout float outVal)
{// declare at the beginning of external parameters
	if (inVal<=inMax && inVal>=inMin)
	{
		outVal = clamp(inVal, inMin, inMax);
		float inScaled = (outVal-inMin)/(inMax-inMin);
		float inCurved = pow(inScaled, curve);
		outVal = inCurved*(outMax-outMin)+outMin;
	};
}

If it looks scary to you, don't be. To be a good ENB tweaker you don't have to deal with tone mapping/adaptation all the time. For most it's good enough to understand how they work. Just copy paste the code of other shader programmers--with their permissions of course

Similarly, above method can be used on gamma, to make image brighter in darkness, and darker under sun light, simulating how human eyes work. Or use it on bloom, making bloom intensity adjust with brightness of the scene etc. All you need is imagination.

Finally:

How adaptation+tone mapping look like:

Declare before Pixel Shader

Code: Select all

void getCurve(float inVal, float inMin, float inMax, float outMin, float outMax, float curve, inout float outVal)
{// declare at the beginning of external parameters
	if (inVal<=inMax && inVal>=inMin)
	{
		outVal = clamp(inVal, inMin, inMax);
		float inScaled = (outVal-inMin)/(inMax-inMin);
		float inCurved = pow(inScaled, curve);
		outVal = inCurved*(outMax-outMin)+outMin;
	};
}

Inside pixel shader

Code: Select all

float lumaWhite = 1.08;
	getCurve(gadapt, 0.2,2.0, 1.1,4.0, 1.4, lumaWhite);
	getCurve(gadapt, 2.0,100, 4.0,4.0, 1, lumaWhite);
	
	// Tone map
	float lumaIn = dot(color.xyz,float3(0.27, 0.67, 0.06));
	float lumaNew = lumaIn*(1+lumaIn/pow(lumaWhite,2))/(1+lumaIn);
	color.xyz *= lumaNew/lumaIn;

In above code gadapt is the adaptation luminance value. lumaWhite is the White Point/exposure value/max range.

Note that I use lumaIn, lumaNew etc. rather than put color.xyz directly in the tone mapping formula. I'll explain this next time in color manipulation.