Weird hash around textures

fixing bugs
Post Reply
  • Author
  • Message
Offline
User avatar
Posts: 27
Joined: 05 Jan 2016, 08:14

Weird hash around textures

I've been working on another ENB (Fallout 3, v0.278) and this time I decided to enable APPLYGAMECOLORCORRECTION. When I did, I started seeing hash marks around objects that reflect or emit light. Here are two shots of the moon from Fallout 3, the first is with color correction enabled (note the hashes) and the second is with it disabled (hashes are gone).

Image

Image

Has anyone encountered this before? And if so, how did you fix it (other than disabling color correction)?

I'm using an AMD card and I've never had a problem with it before. Is this something unique to AMD cards? Or maybe its something in an ini file?

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17427
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: Weird hash around textures

It's not AMD related issue, but have no idea what causing it, how color correction can affect or produce it.
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
User avatar
*blah-blah-blah maniac*
Posts: 3123
Joined: 27 Jan 2012, 13:42

Re: Weird hash around textures

Low-res bloom?

Offline
User avatar
Posts: 27
Joined: 05 Jan 2016, 08:14

Re: Weird hash around textures

Thought that about bloom. I did add a line to preserve ENB bloom and adjusted the existing line accordingly (commented out). But even with ENB bloom disabled it is still there. I have not tried it with a custom bloom shader, though.

Here is what I've tested thus far to try and find the source of the problem:
  • 1. Updated vid card drivers to latest release (vid card is R9 390 w/ 8GB)
  • 2. Rolled back vid card drivers to crimson, then pre-crimson
  • 3. Used unmodified fallout.ini and falloutprefs.ini files
  • 4. Disabled/enabled compression, x64, fix transparency bugs in enblocal.ini
  • 5. Commented out all changes made to enbeffect.fx file, one change at a time
  • 6. Disabled ENB bloom
Regardless of what changes I make as outlined above, the problem persists as long as I have color correction enabled. :?

================

Spent the afternoon checking everything, including the palettes I've made. I'm using HDR in the shader code. Is it that you shouldn't use color correction with HDR?

Offline
User avatar
*blah-blah-blah maniac*
Posts: 3123
Joined: 27 Jan 2012, 13:42

Re: Weird hash around textures

Have you tested whether you can make the issue visible without using AGCC by increasing brightness, modifying gamma etc.?

Also, can you look up the line where original bloom is applied (if there's one, I think there should be - never played FO3)?

Offline
User avatar
Posts: 27
Joined: 05 Jan 2016, 08:14

Re: Weird hash around textures

@mindflux

Hi! While I don't mean to be tedious in my reply, here's where I'm at with the enbeffect.fx file. When I started putting it together I tried to keep to the 'less is more' philosophy (I'm still learning about shaders). I started with expanding on some work Boris did a few years back when he separated LDR controllers into night and day. I simply added LDR for interior, and then the gui spinners. Here's the code:

Code: Select all

float 	 ECCBrightness=lerp( lerp(ECCBrightnessNight, ECCBrightnessDay, ENightDayFactor), ECCBrightnessInterior, EInteriorFactor);
float   ECCGamma=lerp( lerp(ECCGammaNight, ECCGammaDay, ENightDayFactor), ECCGammaInterior, EInteriorFactor);
float 	 ECCSaturation=lerp( lerp(ECCSaturationNight, ECCSaturationDay, ENightDayFactor), ECCSaturationInterior, EInteriorFactor);
float 	 ECCInBlack=lerp( lerp(ECCInBlackNight, ECCInBlackDay, ENightDayFactor), ECCInBlackInterior, EInteriorFactor);
float 	 ECCOutBlack=lerp( lerp(ECCOutBlackNight, ECCOutBlackDay, ENightDayFactor), ECCOutBlackInterior, EInteriorFactor);
float 	 ECCContrast=lerp( lerp(ECCContrastNight, ECCContrastDay, ENightDayFactor), ECCContrastInterior, EInteriorFactor);
float 	 ECCInWhite=lerp( lerp(ECCInWhiteNight, ECCInWhiteDay, ENightDayFactor), ECCInWhiteInterior, EInteriorFactor);
float 	 ECCOutWhite=lerp( lerp(ECCOutWhiteNight, ECCOutWhiteDay, ENightDayFactor), ECCOutWhiteInterior, EInteriorFactor);
float 	 ECCContrastGrayLevel=lerp( lerp(ECCContrastGrayLevelNight, ECCContrastGrayLevelDay, ENightDayFactor), ECCContrastGrayLevelInterior, EInteriorFactor);
float 	 ECCDesaturateShadows=lerp( lerp(ECCDesaturateShadowsNight, ECCDesaturateShadowsDay, ENightDayFactor), ECCDesaturateShadowsInterior, EInteriorFactor);
float3   ECCChannelMixerR=lerp( lerp(ECCChannelMixerRNight, ECCChannelMixerRDay, ENightDayFactor), ECCChannelMixerRInterior, EInteriorFactor);
float3   ECCChannelMixerG=lerp( lerp(ECCChannelMixerGNight, ECCChannelMixerGDay, ENightDayFactor), ECCChannelMixerGInterior, EInteriorFactor);
float3   ECCChannelMixerB=lerp( lerp(ECCChannelMixerBNight, ECCChannelMixerBDay, ENightDayFactor), ECCChannelMixerBInterior, EInteriorFactor);
float3   ECCColorBalanceHighlights=lerp( lerp(ECCColorBalanceHighlightsNight, ECCColorBalanceHighlightsDay, ENightDayFactor), ECCColorBalanceHighlightsInterior, EInteriorFactor);
float3   ECCColorBalanceShadows=lerp( lerp(ECCColorBalanceShadowsNight, ECCColorBalanceShadowsDay, ENightDayFactor), ECCColorBalanceShadowsInterior, EInteriorFactor);
That was easy enough, so I did the same to the posprocess code and added gui spinners for that as well:

Code: Select all

float newEAdaptationMinV2=lerp( lerp(EAdaptationMinV2Night, EAdaptationMinV2Day, ENightDayFactor), EAdaptationMinV2Interior, EInteriorFactor ); 
float newEAdaptationMaxV2=lerp( lerp(EAdaptationMaxV2Night, EAdaptationMaxV2Day, ENightDayFactor), EAdaptationMaxV2Interior, EInteriorFactor ); 
float newEBrightnessV2=lerp( lerp(EBrightnessV2Night, EBrightnessV2Day, ENightDayFactor), EBrightnessV2Interior, EInteriorFactor );
float newEToneMappingOversaturationV2=lerp( lerp(EToneMappingOversaturationV2Night, EToneMappingOversaturationV2Day, ENightDayFactor), EToneMappingOversaturationV2Interior, InteriorFactor );
float newEToneMappingCurveV2=lerp( lerp(EToneMappingCurveV2Night, EToneMappingCurveV2Day, ENightDayFactor), EToneMappingCurveV2Interior, EInteriorFactor );	
float newEColorSaturationV2=lerp( lerp(EColorSaturationV2Night, EColorSaturationV2Day, ENightDayFactor), EColorSaturationV2Interior, EInteriorFactor );
float newEIntensityContrastV2=lerp( lerp(EIntensityContrastV2Night, EIntensityContrastV2Day, ENightDayFactor), EIntensityContrastV2Interior, EInteriorFactor );
Both sets of interpolators are at the top of the pixel shader section (inside the opening bracket and before float4 _oC0=0.0;)

Finally, I added a snippet of code that Prod80 posted (Filmic HDR). That was placed between postprocess and enbpalette code. Added the spinners, and everything was fine, worked like a charm.

Then I decided to activate color correction and the moon turned into a Belgian waffle - as did interior lights, and items that reflect light.

Since you mentioned not messing with Fallout 3 or New Vegas, the bloom code is the same for Skyrim (which I also play - perfect time to thank you for your Skyrim patches!).

As for bloom. JawZ put up a bit of code (viewtopic.php?) that allows the use of ENB bloom when color correction is enabled. Brilliant bit of code, and it works like a charm. But that's not the problem.

I thought as you and removed the code for ENB bloom since color correction was enabled. Problem is still there. I also removed the adjusted enbeffect ini file to see if it was a gamma problem. Not only was the moon still a waffle, it started dripping syrup (the hash/waffle effect was very clear).

So, I'm starting to think that color correction just doesn't play well with modifications to the base shader code. Why? I dunno. I'm still new to shaders and it took me a while to get this far. :D And, given how old the game is I doubt Boris is inclined to jump on this with the intent to work it out.

And that's where I'm at. Thanks for the suggestions! If you want, I can zip up the complete shader file and post it. Maybe I'm missing something (a bit of code)?

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17427
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: Weird hash around textures

Shout in the dark: dithering is enabled somewhere or offsets for reading several samples? In later shaders effect.txt for example. Or code there for sharpening. I can't believe that color manipulations can do this problem. Did you tried updated recently color correction code of enbeffect.fx, are you sure?
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
User avatar
Posts: 27
Joined: 05 Jan 2016, 08:14

Re: Weird hash around textures

I figured it out. I took copies of the shader file and ran a comparison in N++. Turns out the problem started with the fella fingering the keyboard (me). Dyslexia and lack of sleep is a bitch. :oops:

Specifically, it was in samplers section. I reduced each sampler to a single line and rolled right over the fact that _s0 min/magfilter is set to point and not linear.
Post Reply