Problem with my Box-Blur code

[Euda]

Hello Community

TL;DR-version:
- having two techniques: 1st: Horizontal blur of original image; 2nd: Vertical blur of 1st-technique's result => Box Blur
- now I want to add the resulting box blurred image to the original screen. Problem: return origcolor+result*0.3 won't work because origcolor contains the result of pass 1 instead of the original image
-> question: How to sample the original, unmodified pixel when being in a second/third/fourth/... pass? Big thanks in advance and best regards,
Euda


I'm currently trying to acquire some simple skills in writing effect-files for (mainly GTA San Andreas 0.076+) ENBSeries. So I wanted to create a simple bloom effect, based on a box blur added to the original image. Wrote it as a single technique and visually it worked fine whereas the framerate dropped by a vast amount when exceeding ~20px of sampling-radius for the blur (AMD R9 290X).

Here's a screenshot of the result (notice the framerate, sample radius was 64px):

http://image-share.de/images/fd5ce071e8 ... bf344c.png
__
Today - after arriving from a short vacation - I split my code into two passes. The first pass blurs the original image horizontally, the second one blurs the result of pass 1 vertically.
After saving in Notepad++ and alt-tabbing into the game, I recognized a huge fps-gain over my previous method:

http://image-share.de/images/1201d246b3 ... b38cce.png
respectively (+MRTRendering, -UseWater): http://image-share.de/images/a3fb997754 ... 19f7de.png
__

Subsequently, to "transform" that method from a blur-shader into a bloom-shader, I'd be required to add the blurred result to the original, pre-postprocessing-image - which is when my problem kicked in

I'm lacking any idea on how to sample a pixel of the original image in my second pass.
To simply add the blur by a low percentage instead of completely returning it to the screen, I'd have to output the original image in my second pass + the blur-result * 0.3 for example. Tried this, but it literally sampled a pixel of the horizontally blurred image instead of one of the original image without any blur.

Here are the two effect-files (San Andreas ENBSeries 0.076), single- and multipass:

Code: Select all

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// ENBSeries effect file
// http://enbdev.com
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//THIS IS HLSL FILE FORMAT FOR EXECUTING ADDITIONAL
//POST PROCESSING EFFECTS. MAKE THE COPY BEFORE CHANGING IT!
//WARNING! This is new standart file, compatible with ENBSeries 0.101 or better

//+++++++++++++++++++++++++++++
//external parameters, do not modify
//+++++++++++++++++++++++++++++
//keyboard controlled temporary variables (in some versions exists in the config file). Press and hold key 1,2,3...8 together with PageUp or PageDown to modify. By default all set to 1.0
float4	tempF1; //0,1,2,3
float4	tempF2; //5,6,7,8
float4	tempF3; //9,0
//x=generic timer in range 0..1, period of 16777216 ms (4.6 hours), w=frame time elapsed (in seconds)
float4	Timer;
//x=Width, y=1/Width, z=ScreenScaleY, w=1/ScreenScaleY
float4	ScreenSize;


texture2D texColor;	
texture2D texDepth;	
texture2D texNoise;	

sampler2D SamplerColor = sampler_state
{
	Texture   = <texColor>;
	MinFilter = LINEAR;
	MagFilter = LINEAR;
	MipFilter = NONE;
	AddressU  = Clamp;
	AddressV  = Clamp;
	SRGBTexture=FALSE;
	MaxMipLevel=0;
	MipMapLodBias=0;
};

sampler2D SamplerDepth = sampler_state
{
	Texture   = <texDepth>;
	MinFilter = POINT;
	MagFilter = POINT;
	MipFilter = NONE;
	AddressU  = Clamp;
	AddressV  = Clamp;
	SRGBTexture=FALSE;
	MaxMipLevel=0;
	MipMapLodBias=0;
};

sampler2D SamplerNoise = sampler_state
{
	Texture   = <texNoise>;
	MinFilter = POINT;
	MagFilter = POINT;
	MipFilter = NONE;
	AddressU  = Wrap;
	AddressV  = Wrap;
	SRGBTexture=FALSE;
	MaxMipLevel=0;
	MipMapLodBias=0;
};


struct VS_OUTPUT_POST
{
	float4 vpos  : POSITION;		
	float2 txcoord : TEXCOORD0;
};

struct VS_INPUT_POST
{
	float3 pos  : POSITION;
	float2 txcoord : TEXCOORD0;
};



//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// Vertex-Shader
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
VS_OUTPUT_POST VS_PostProcess(VS_INPUT_POST IN)
{
	VS_OUTPUT_POST OUT;

	float4 pos=float4(IN.pos.x,IN.pos.y,IN.pos.z,1.0);

	OUT.vpos=pos;
	OUT.txcoord.xy=IN.txcoord.xy;

	return OUT;
}

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// Pixel-Shader
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

float4 PS_Process(VS_OUTPUT_POST IN, float2 vPos : VPOS) : COLOR
{
	float4	res;	// result
	float4  origcolor = tex2D(SamplerColor, IN.txcoord.xy);
	res.xyz = origcolor.xyz;
	res.w = 1.0;
	float2 actPos = IN.txcoord.xy;
	float2 posi = actPos;

	float3 samples;
	float multip = 1;	// skipped samples per loop-cycle
	int sc = 64;   	// sample count => quality
	
		for (float m=1; m<sc; m++)
		{
			for (float j=1; j<sc; j++)
			{				
				posi=actPos;
				posi.y += multip * m * ScreenSize.y;
				posi.x += multip * j * ScreenSize.y;
				samples.xyz+=tex2D(SamplerColor, posi.xy);
				posi=actPos;
				posi.y += multip * m * ScreenSize.y;
				posi.x -= multip * j * ScreenSize.y;
				samples.xyz+=tex2D(SamplerColor, posi.xy);
				posi=actPos;
				posi.y -= multip * m * ScreenSize.y;
				posi.x += multip * j * ScreenSize.y;
				samples.xyz+=tex2D(SamplerColor, posi.xy);
				posi=actPos;
				posi.y -= multip * m * ScreenSize.y;
				posi.x -= multip * j * ScreenSize.y;
				samples.xyz+=tex2D(SamplerColor, posi.xy);			
			}
		}
						
	samples.xyz /= sc*sc*2;
	res.xyz = samples.xyz;
		
	return res;
}



//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
technique PostProcess
{
	pass P0
	{
	
		VertexShader = compile vs_3_0 VS_PostProcess();
		PixelShader  = compile ps_3_0 PS_Process();

		DitherEnable=FALSE;
		ZEnable=FALSE;
		CullMode=NONE;
		ALPHATESTENABLE=FALSE;
		SEPARATEALPHABLENDENABLE=FALSE;
		AlphaBlendEnable=FALSE;
		StencilEnable=FALSE;
		FogEnable=FALSE;
		SRGBWRITEENABLE=FALSE;
	}
}

____________________________________________

Code: Select all

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// ENBSeries effect file
// http://enbdev.com
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//WARNING! This is new standart file, compatible with ENBSeries 0.101 or better

//+++++++++++++++++++++++++++++
//external parameters, do not modify
//+++++++++++++++++++++++++++++
//keyboard controlled temporary variables (in some versions exists in the config file). Press and hold key 1,2,3...8 together with PageUp or PageDown to modify. By default all set to 1.0
float4	tempF1; //0,1,2,3
float4	tempF2; //5,6,7,8
float4	tempF3; //9,0
//x=generic timer in range 0..1, period of 16777216 ms (4.6 hours), w=frame time elapsed (in seconds)
float4	Timer;
//x=Width, y=1/Width, z=ScreenScaleY, w=1/ScreenScaleY
float4	ScreenSize;


texture2D texColor;	
texture2D texDepth;	
texture2D texNoise;	

sampler2D SamplerColor = sampler_state
{
	Texture   = <texColor>;
	MinFilter = LINEAR;
	MagFilter = LINEAR;
	MipFilter = NONE;
	AddressU  = Clamp;
	AddressV  = Clamp;
	SRGBTexture=FALSE;
	MaxMipLevel=0;
	MipMapLodBias=0;
};

sampler2D SamplerDepth = sampler_state
{
	Texture   = <texDepth>;
	MinFilter = POINT;
	MagFilter = POINT;
	MipFilter = NONE;
	AddressU  = Clamp;
	AddressV  = Clamp;
	SRGBTexture=FALSE;
	MaxMipLevel=0;
	MipMapLodBias=0;
};

sampler2D SamplerNoise = sampler_state
{
	Texture   = <texNoise>;
	MinFilter = POINT;
	MagFilter = POINT;
	MipFilter = NONE;
	AddressU  = Wrap;
	AddressV  = Wrap;
	SRGBTexture=FALSE;
	MaxMipLevel=0;
	MipMapLodBias=0;
};


struct VS_OUTPUT_POST
{
	float4 vpos  : POSITION;		
	float2 txcoord : TEXCOORD0;
};

struct VS_INPUT_POST
{
	float3 pos  : POSITION;
	float2 txcoord : TEXCOORD0;
};

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// Vertex-Shader
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

VS_OUTPUT_POST VS_PostProcess(VS_INPUT_POST IN)
{
	VS_OUTPUT_POST OUT;

	float4 pos=float4(IN.pos.x,IN.pos.y,IN.pos.z,1.0);

	OUT.vpos=pos;
	OUT.txcoord.xy=IN.txcoord.xy;

	return OUT;
}

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// Pixel-Shader || Horizontal Blur
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

float4 PS_Process(VS_OUTPUT_POST IN, float2 vPos : VPOS) : COLOR
{
	float4	res;	// result
	float4  origcolor = tex2D(SamplerColor, IN.txcoord.xy);
	res.w = 1.0;
	float2 actPos = IN.txcoord.xy;
	float2 posi = actPos;
	
	float3 samples;
	float multip = 1;	// skipped samples per loop-cycle
	int sc = 64;   	// sample amount => quality
	
		for (float j=1; j<sc; j++)
		{				
			posi=actPos;
			posi.x += multip * j * ScreenSize.y;
			samples.xyz+=tex2D(SamplerColor, posi.xy);
			posi=actPos;
			posi.x -= multip * j * ScreenSize.y;		
			samples.xyz+=tex2D(SamplerColor, posi.xy);
		}
						
	samples.xyz /= sc;
	res.xyz = samples.xyz;
	
	// todo: VIBRANCE!
	
	int compareMode = 0;		// splitscreen comparison
	if (compareMode == 1) 
	{
		if (actPos.x < 0.5) return origcolor;
		else return res;
	}
	else return res;
}

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// Pixel-Shader || Vertical Blur
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

float4 PS_Passb(VS_OUTPUT_POST IN, float2 vPos : VPOS) : COLOR
{
	float4	res;	// result
	float4  origcolor = tex2D(SamplerColor, IN.txcoord.xy);
	res.w = 1.0;
	float2 actPos = IN.txcoord.xy;
	float2 posi = actPos;

	float3 samples;
	float multip = 1;	// skipped samples per loop-cycle
	int sc = 64;   	// sample count => quality
	
		for (float j=1; j<sc; j++)
		{				
			posi=actPos;
			posi.y += multip * j * ScreenSize.y;
			samples.xyz+=tex2D(SamplerColor, posi.xy);
			posi=actPos;
			posi.y -= multip * j * ScreenSize.y;		
			samples.xyz+=tex2D(SamplerColor, posi.xy);
		}
						
	samples.xyz /= sc*2;
	res.xyz = samples.xyz;
	
	
	int compareMode = 0;		// splitscreen comparison
	if (compareMode == 1) 
	{
		if (actPos.x < 0.5) return origcolor;
		else return res;
	}
	else return res;
[b]// else return origcolor+res*0.3 --> That's my problem - origcolor would have to be a sample of the original pixel, rather than a sample of PS_Process' result: A horizontal blurred pixel. :/[/b]

}

//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// Techniques
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
technique PostProcess
{
	pass P0
	{
	
		VertexShader = compile vs_3_0 VS_PostProcess();
		PixelShader  = compile ps_3_0 PS_Process();

		DitherEnable=FALSE;
		ZEnable=FALSE;
		CullMode=NONE;
		ALPHATESTENABLE=FALSE;
		SEPARATEALPHABLENDENABLE=FALSE;
		AlphaBlendEnable=FALSE;
		StencilEnable=FALSE;
		FogEnable=FALSE;
		SRGBWRITEENABLE=FALSE;
	}
}

technique PostProcess2
{
	pass P0
	{
	
		VertexShader = compile vs_3_0 VS_PostProcess();
		PixelShader  = compile ps_3_0 PS_Passb();

		DitherEnable=FALSE;
		ZEnable=FALSE;
		CullMode=NONE;
		ALPHATESTENABLE=FALSE;
		SEPARATEALPHABLENDENABLE=FALSE;
		AlphaBlendEnable=FALSE;
		StencilEnable=FALSE;
		FogEnable=FALSE;
		SRGBWRITEENABLE=FALSE;
	}
}

__

Finally, I hope that I've provided you guys with enough information and described my issue as precise as possible, I'm thankful for any help and currently idealess after 2 hours of testing and combining different ways to fix my problem.
Thanks in advance,
Euda

[prod80]

have a look in this effect.txt file:
viewtopic.php?f=7&t=3189

it's sharpening, but as you probably know, sharpening also works with blurring.

In any case, you need to add texOriginal, which contains the original texture (output of enbeffect.fx) and isn't changed during the passes. texColor changes in each pass. That way you can use 2 passes for horizontal and vertical blurs, than add it to texOriginal, or do with it as you please. In the effect.txt file linked I add the result of 5 passes at the end of the 5th pass to the original. There is no way to retain the original texture (before first pass) and use that in the second or consecutive passes unless you write it to a texture, which we unfortunately can't do in ENB.

- I do not know if effect.txt features texOriginal in other binaries besides Skyrim, doesn't hurt to try. If it doesn't work then maybe Boris can add (if he wants) or you need to do everything in a single pass... which costs quite a bit more performance (as you know). However you can greatly optimize blurring by using lookup arrays and a process called "linear sampling". Have a look here for some info: http://rastergrid.com/blog/2010/09/effi ... -sampling/ (it's for Gaussian, but you can just as well apply this method to everything else).
- this line: "for (float j=1; j<sc; j++)" in your loops. Doesn't this always cause the loop to unroll? See if performance increases when you simply change it to "for (float j=1; j<64; j++)". And, 64 samples is a lot. A lot too much for bloom (imo), but depends on what you want to create.
- Dunno that binary, but doesn't it support enbbloom.fx?

Have fun

[Marty McFly]

Prod80, that binary is for GTASA so you naturally don't know it. Unfortunately it does not support texOriginal, that is why I had to create bloom by downscaling image, putting the r g and b channels on different screen areas of the alpha channel, blur thr alpha channel in the next two passes, and upscale and merge the r g and b areas of alpha channel again. Hacky method but works even on old generic ENB versions.

[kingeric1992]

Marty McFly, you just solved my problem! I was trying to do some separable shaped bokeh dof which need buffer to accumulate the result, I posted it in offtopic-general. Thanks.
viewtopic.php?f=17&t=3277

[Euda]

In any case, you need to add texOriginal, which contains the original texture (output of enbeffect.fx) and isn't changed during the passes. texColor changes in each pass. That way you can use 2 passes for horizontal and vertical blurs, than add it to texOriginal, or do with it as you please. In the effect.txt file linked I add the result of 5 passes at the end of the 5th pass to the original. There is no way to retain the original texture (before first pass) and use that in the second or consecutive passes unless you write it to a texture, which we unfortunately can't do in ENB.

- I do not know if effect.txt features texOriginal in other binaries besides Skyrim, doesn't hurt to try. If it doesn't work then maybe Boris can add (if he wants) or you need to do everything in a single pass... which costs quite a bit more performance (as you know). However you can greatly optimize blurring by using lookup arrays and a process called "linear sampling". Have a look here for some info: http://rastergrid.com/blog/2010/09/effi ... -sampling/ (it's for Gaussian, but you can just as well apply this method to everything else).
- this line: "for (float j=1; j<sc; j++)" in your loops. Doesn't this always cause the loop to unroll? See if performance increases when you simply change it to "for (float j=1; j<64; j++)". And, 64 samples is a lot. A lot too much for bloom (imo), but depends on what you want to create.
- Dunno that binary, but doesn't it support enbbloom.fx?

Have fun

At first, big thanks for your help!
The shader was written for and tested with GTA San Andreas' 0.076 binary which, as Marty stated, doesn't store the original pixel in a texture (as texOriginal for example, if defined in the effect-file). Unfortunately the same situation for GTA IV 0.163-binary so I kept on that singlepass-bloom which is really slow (x² vs. 2x tex-fetches). However, I managed the effect to strongly benefit from the information about linear sampling in your link - so again, Kudos for doubling the performance prod!
That was a few days ago, and ongoing, I downloaded Skyrim again and threw the "RealVision"-ENB together with my bloom-effect in the game-folder. I further transformed the shader from single- to multipass (+ linear sampling) and was glad to see the huge performance-gain. Currently, I've provided four modifiable parameters:

Sample-Count (Sampling radius in pixels, 512 costs like 10% fps for me, 128px is almost for free)
Skipped Samples (specifies how many pixels get skipped per taken sample, decreases performance-cost but increases some ugly banding-artifact, resulting bloom-radius is skippedSamples*samples)
Bloom Strength (kinda self-explaining :>)
Luma Damping (damps the affection of bright pixels, thus bright areas won't turn partially white - softly reduces overall contrast though)

@Marty: Sadly, I don't understand how you managed to trick the original-pixel into the 2nd/3rd+ passes. I just took a look over your recent mastereffect v1.5 and assume it's that "texture to texture"-stuff in there. Astonishing work though, screenshots @ GTAF are coming soon
_

Here are some comparison shots with and without my bloom (settings were: Radius 64; Skipped Samples 0; Luma Damping 0.8 (80%); Bloom Strength 0.9):














Euda

[prod80]

@Euda
Now try and use texDepth in your bloom ...ie if you see your screenshot of the bridge you see the blooming of the bridge highlights on the water in the background. You can get very nice results by z testing your bloom and depend both the radius and area's to bloom on the depth relation between pixels.

Here's my bloom shader, so you get an idea what I mean

[Marty McFly]

@Euda: it is actually a simple but extremely effective way to fake downsampling, as all SA versions lack tex2dlod support (you can use it but mipmap leves higher than 0 have no effect, image stays fullscreen). I used texcoord modifications to scale the image down in screenspace (so it looks like a smaller image in a black surrounding area). Now by adding/subtracting different values to he texcoord the smaller versions of the fullscreen image are on different areas on screen per channel - the miniature image of the red channel is on top left corner of screen, green one is a bit lower and right). So I can save r g and b in only 1 channel, the alpha channel. In the next 2 passes only the alpha channel gets blurred, nothing else. Now in the 4th pass, the texcoords get manipulated again to stretch and shift the miniature images back to fullscreen. Result is downscaled image, blurred vertically and horizontally with gaussian weighting. This can now be used to blend with original image.

Pass 1: downscale image, put miniature versions of r g and b channel image on different screen areas of alpha channel. Thresholding.

Pass 2 and 3: vertical and horizontal gaussian blur of alpha channel only

Pass 4: upscale the r g and b miniature versions again and put them together. Mix with original image.

I am extremely proud of my LDR mixing method, it does not destroy contrast and does not produce pure whites. Couldn't be better.


@prod80: Since I'm gonna release my ENB soon, may I have your permission to use large parts of your enbbloom.fx code? Basically most of the gaussian h v blur with qualitry control is used so bloompass1 and 2 are almost completely your work. Credits are given, of course.

[prod80]

Use as you see fit, but it's based on this work: http://http.developer.nvidia.com/GPUGem ... _ch40.html which I assume is Adobe's Gaussian blur technique, since its written by an Adobe developer.

If you take out the depth testing it's an extremely efficient blurring method which on top of it all is highly configurable.