ENB Debanding

[grvulture]

This is a conversion of ReShade's debanding shader. It works well, until the sunset transitions occur (in Skyrim SE where is was tested), then it bugs out, and I couldn't fix it, so I gave up on it. I'm sharing it here in case anybody else wants to fiddle with it and maybe make it work properly.

Requirements are that your gamma and color correction are (somewhat) correct, otherwise there is no way to deband properly.

The code is to be added to enbeffect.fx. It takes your LUT texture as input, so you have to call the shader with it, eg.:

Code: Select all

Texture2D          LUT_default < string UIName = "3DLut0";  string ResourceName = "Vulture ENB.png"; >; // default Lut
technique11 Draw <string UIName="Vulture ENB";>
{
   pass p0
   {
      SetVertexShader(CompileShader(vs_5_0, VS_Draw()));
      SetPixelShader(CompileShader(ps_5_0, PS_Draw(LUT_default)));
   }
}

and your PS_Draw should be declared like this:

Code: Select all

float4  PS_Draw(VS_OUTPUT_POST IN, float4 v0 : SV_Position0, uniform Texture2D LUTtex) : SV_Target

I tried to add some images in imgur to see it in action but jpg compression re-bands the images so it's not possible.

GUI:

Code: Select all

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//Debanding by grvulture UI
float  empty46          <string UIName="|===== Debanding =====|";        string UIWidget="spinner";         float UIMin=0.0;  float UIMax=0.0;     float UIStep=0.0; >  = {0.0}; 
int   Debanding         <string UIName ="----------Debanding adjustment---------";string UIWidget="spinner";int UIMin=0;      int UIMax=0;>                             = {0.0};
bool  DBEnable          <string UIName = "Use Debanding";>                                                                                                              = {true};
float DBThreshold       <string UIName = "Debanding Threshold";		     string UIWidget="Spinner";         float UIMin=0.0;  float UIMax=0.5;     float UIStep=0.001;> = {0.025};
float DBRange           <string UIName = "Debanding Range";              string UIWidget="Spinner";         float UIMin=0;    float UIMax=32768;   float UIStep=1;>     = {8192};
int   DBIterations      <string UIName = "Debanding Iterations";         string UIWidget="Spinner";         float UIMin=0;    float UIMax=10;      float UIStep=1;>     = {2};
float DBGrain           <string UIName = "Debanding Noise";		         string UIWidget="Spinner";         float UIMin=0.0;  float UIMax=0.5;     float UIStep=0.001;> = {0.075};
int   DebandColors      <string UIName ="-------Debanding color adjustments-----";string UIWidget="spinner";int UIMin=0;      int UIMax=0;>                             = {0.0};
float DBredAddSunrise   <string UIName = "Red Deband Shift - Sunrise";   string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {0.00};
float DBredAdd          <string UIName = "Red Deband Shift - Day";       string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {0.01};
float DBredAddSunset    <string UIName = "Red Deband Shift - Sunset";    string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {-0.07};
float DBgreenAddSunrise <string UIName = "Green Deband Shift - Sunrise"; string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {0.00};
float DBgreenAdd        <string UIName = "Green Deband Shift - Day";     string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {0.00};
float DBgreenAddSunset  <string UIName = "Green Deband Shift - Sunset";  string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {0.00};
float DBblueAddSunrise  <string UIName = "Blue Deband Shift - Sunrise";  string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {0.00};
float DBblueAdd         <string UIName = "Blue Deband Shift - Day";      string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {0.01};
float DBblueAddSunset   <string UIName = "Blue Deband Shift - Sunset";   string UIWidget="Spinner";         float UIMin=-1.0; float UIMax=1.0;     float UIStep=0.01;>  = {-0.01};
float  empty48          <string UIName="|=====================|";        string UIWidget="spinner";         float UIMin=0.0;  float UIMax=0.0;     float UIStep=0.0; >  = {0.0}; 
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////

FUNCTIONS:

Code: Select all

//=================================================================
//================= Deband by grvulture FUNCTIONS =================
//= Based on Deband shader by haasn for ReShade. Enriched for ENB =
//=================================================================
float rand(float x)
{
  return frac(x * 0.024390243);
}
float permute(float x)
{
  return ((34.0 * x + 1.0) * x) % 289.0;
}
float3 average(uniform Texture2D LUTtex, float2 pos, float range, inout float h)
{
  // Helper: Compute a stochastic approximation of the avg color around a pixel

  // Compute a random range and distance
  float dist = rand(h) * range;     h = permute(h);
  float dir  = rand(h) * 6.2831853; h = permute(h);

  float2 pt = dist * PixelSize;
  float2 o = float2(cos(dir), sin(dir));

  // Sample at quarter-turn intervals around the source pixel
  float3 ref[4];
  ref[0] = LUTtex.SampleLevel(Sampler0, pos + pt * float2( o.x,  o.y), 0).rgb;
  ref[1] = LUTtex.SampleLevel(Sampler0, pos + pt * float2(-o.y,  o.x), 0).rgb;
  ref[2] = LUTtex.SampleLevel(Sampler0, pos + pt * float2(-o.x, -o.y), 0).rgb;
  ref[3] = LUTtex.SampleLevel(Sampler0, pos + pt * float2( o.y, -o.x), 0).rgb;

  // Return the (normalized) average
  return (ref[0] + ref[1] + ref[2] + ref[3]) * 0.25;
}
//=================================================================
//=================================================================

SHADER:

Code: Select all

  //=================================================================
  //================= Deband by grvulture SHADER ====================
  //= Based on Deband shader by haasn for ReShade. Enriched for ENB =
  //=================================================================
  if (DBEnable && EInteriorFactor==0) {
	float   timevalue;
	float   timeweight;
	timeweight=0.000001;
	timeweight+=TimeOfDay1.x;
	timeweight+=TimeOfDay1.y;
	timeweight+=TimeOfDay1.z;
	timeweight+=TimeOfDay1.w;
	timeweight+=TimeOfDay2.x;
	timeweight+=TimeOfDay2.y;

    // Initialize the PRNG by hashing the position + a tiny uniform
    float3 m = (0.1, 0.1, 0.1);
    float h = permute(permute(permute(m.x) + m.y) + m.z);

    // Sample the source pixel
    float3 avg;
    float3 diff;
    
    for (int i = 1; i <= DBIterations; i++)
    {
      // Sample the average pixel and use it instead of the original if the difference is below the given threshold
      avg = average(LUTtex, coord, i * DBRange, h);
      diff = abs(color.rgb - avg);
      color.rgb = lerp(avg+(diff/2), color.rgb, diff > DBThreshold * i);

      timevalue=0.0;
        
      timevalue+=TimeOfDay1.x * 0;
      timevalue+=TimeOfDay1.y * DBredAddSunrise;
      timevalue+=TimeOfDay1.z * DBredAdd;
      timevalue+=TimeOfDay1.w * DBredAddSunset;
      timevalue+=TimeOfDay2.x * 0;
      timevalue+=TimeOfDay2.y * 0;

      float newDBredAdd;
      newDBredAdd=lerp( (timevalue / timeweight), 0, EInteriorFactor );

      timevalue=0.0;
        
      timevalue+=TimeOfDay1.x * 0;
      timevalue+=TimeOfDay1.y * DBgreenAddSunrise;
      timevalue+=TimeOfDay1.z * DBgreenAdd;
      timevalue+=TimeOfDay1.w * DBgreenAddSunset;
      timevalue+=TimeOfDay2.x * 0;
      timevalue+=TimeOfDay2.y * 0;

      float newDBgreenAdd;
      newDBgreenAdd=lerp( (timevalue / timeweight), 0, EInteriorFactor );

      timevalue=0.0;
        
      timevalue+=TimeOfDay1.x * 0;
      timevalue+=TimeOfDay1.y * DBblueAddSunrise;
      timevalue+=TimeOfDay1.z * DBblueAdd;
      timevalue+=TimeOfDay1.w * DBblueAddSunset;
      timevalue+=TimeOfDay2.x * 0;
      timevalue+=TimeOfDay2.y * 0;

      float newDBblueAdd;
      newDBblueAdd=lerp( (timevalue / timeweight), 0, EInteriorFactor );
    
      color.r+= newDBredAdd;
      color.g+= newDBgreenAdd;
      color.b+= newDBblueAdd;

      if (DBGrain > 0.0)
      {
        timevalue=0.0;
          
        timevalue+=TimeOfDay1.x * 0;
        timevalue+=TimeOfDay1.y * DBGrainSunrise;
        timevalue+=TimeOfDay1.z * DBGrain;
        timevalue+=TimeOfDay1.w * DBGrainSunset;
        timevalue+=TimeOfDay2.x * 0;
        timevalue+=TimeOfDay2.y * 0;

        float newDBGrain;
        newDBGrain=lerp( (timevalue / timeweight), 0, EInteriorFactor );

        // Add some random noise
        float3 noise;
        noise.x = rand(h); h = permute(h);
        noise.y = rand(h); h = permute(h);
        noise.z = rand(h); h = permute(h);
        color.xyz += newDBGrain * (noise - 0.5);
      }
    }
  }
  //=================================================================
  //=================================================================

The shifting color adjustments shift each resulted color by the amount given and can be useful in certain situations.

[SandvichDISH]

Because ENB gives you access to games's actual HDR buffers before quantisation, you actually don't need a deband shader (which aims to clean up banding after it's already happened). All you need is to insert dithering at the right place, and you can eliminate banding more cleanly than would be possible with reshade! Here's a presentation that explains it in more detail: http://loopit.dk/banding_in_games.pdf

I made a post for the dithering shader I made for my own ENB preset here if you're interested

[Adyss]

That presentation helps a lot! I always was confused about the correct way to do dither. Thanks for the link

[grvulture]

Hey Sandvich, thanks for the explanation and the presentation, man!