enb 0.323 for fallout cant use palettes.

fixing bugs
Post Reply
  • Author
  • Message
Offline
Posts: 6
Joined: 17 Feb 2018, 05:59

enb 0.323 for fallout cant use palettes.

So my problem is that even after following the instructions in the post about using palettes with the dx11 versions of enb series, I cant seem to get it to load my enbpalette.bmp. I get errors upon startup and it does not appear to load enbeffect.fx or the palette although everything else works fine. Are there any updated instructions I missed? Hopefully some one can shed some light on this for me because I am stumped.

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17427
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: enb 0.323 for fallout cant use palettes.

Palettes must be written as code in enbeffect.fx, so obviously the code you or somebody else added is wrong. I didnt any code for that, so how can i help, no idea.
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
Posts: 6
Joined: 17 Feb 2018, 05:59

Re: enb 0.323 for fallout cant use palettes.

sorry if my post was a little hard to understand, I was following your instructions from this post http://enbdev.com/enbseries/forum/viewt ... f=5&t=5387 in which you provided the code to modify enbeffect.fx to be able to use enbpalette.bmp. The problem is after modifying the file (and putting my enbpalette.bmp in the games install folder) upon startup I receive these messages.
G:\steam\steamapps\common\fallout 4\enbseries\enbeffect.fx (322,12): warning x3206: implicit truncation of vector type
G:\steam\steamapps\common\fallout4\enbseries\enbeffect.fx (328,24): errorx3004: undeclared identifier 'gray adaptation'

which seems to cause enbeffect.fx to not load at all.

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17427
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: enb 0.323 for fallout cant use palettes.

Ive updated code on that page, find the following and do changes by removing double slash

Code: Select all

//UNCOMMENT THIS FOR FALLOUT 4
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
Posts: 6
Joined: 17 Feb 2018, 05:59

Re: enb 0.323 for fallout cant use palettes.

Thank you for your prompt reply but it seems I am still getting the same warnings even after updating to your new code using your instructions. I feel like an idiot for not being able to get this working seeing as how your instructions are easy to understand. I have used enb series since tes5 but only using other users presets, this is my first time trying to adjust the base enb files to create my own. Forgive me for being so newbish to this, I'm learning as I go and have been able to find videos and posts with information on everything else I have been tweaking. This is the only thing I am having trouble with and any further help would be greatly appreciated.

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17427
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: enb 0.323 for fallout cant use palettes.

Post your whole code here
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
Posts: 6
Joined: 17 Feb 2018, 05:59

Re: enb 0.323 for fallout cant use palettes.

This is what is in notepad after opening up my modified enbeffect.fx.



//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// ENBSeries Fallout 4 hlsl DX11 format, example post process
// visit http://enbdev.com for updates
// Author: Boris Vorontsov
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

//Warning! In this version Weather index is not yet implemented

//uncomment to use original game post processing
#define APPLYGAMECOLORCORRECTION


//+++++++++++++++++++++++++++++
//internal parameters, modify or add new
//+++++++++++++++++++++++++++++
/*
//example parameters with annotations for in-game editor
float ExampleScalar
<
string UIName="Example scalar";
string UIWidget="spinner";
float UIMin=0.0;
float UIMax=1000.0;
> = {1.0};

float3 ExampleColor
<
string UIName = "Example color";
string UIWidget = "color";
> = {0.0, 1.0, 0.0};

float4 ExampleVector
<
string UIName="Example vector";
string UIWidget="vector";
> = {0.0, 1.0, 0.0, 0.0};

int ExampleQuality
<
string UIName="Example quality";
string UIWidget="quality";
int UIMin=0;
int UIMax=3;
> = {1};

Texture2D PaletteTexture
<
string UIName = "Palette texture";
string ResourceName = "enbpalette.bmp";
>;
SamplerState ExampleSampler
{
Filter = MIN_MAG_MIP_LINEAR;
AddressU = Clamp;
AddressV = Clamp;
};
*/


#ifdef E_CC_PROCEDURAL
//parameters for ldr color correction
float ECCGamma
<
string UIName="CC: Gamma";
string UIWidget="Spinner";
float UIMin=0.2;//not zero!!!
float UIMax=5.0;
> = {1.0};

float ECCInBlack
<
string UIName="CC: In black";
string UIWidget="Spinner";
float UIMin=0.0;
float UIMax=1.0;
> = {0.0};

float ECCInWhite
<
string UIName="CC: In white";
string UIWidget="Spinner";
float UIMin=0.0;
float UIMax=1.0;
> = {1.0};

float ECCOutBlack
<
string UIName="CC: Out black";
string UIWidget="Spinner";
float UIMin=0.0;
float UIMax=1.0;
> = {0.0};

float ECCOutWhite
<
string UIName="CC: Out white";
string UIWidget="Spinner";
float UIMin=0.0;
float UIMax=1.0;
> = {1.0};

float ECCBrightness
<
string UIName="CC: Brightness";
string UIWidget="Spinner";
float UIMin=0.0;
float UIMax=10.0;
> = {1.0};

float ECCContrastGrayLevel
<
string UIName="CC: Contrast gray level";
string UIWidget="Spinner";
float UIMin=0.01;
float UIMax=0.99;
> = {0.5};

float ECCContrast
<
string UIName="CC: Contrast";
string UIWidget="Spinner";
float UIMin=0.0;
float UIMax=10.0;
> = {1.0};

float ECCSaturation
<
string UIName="CC: Saturation";
string UIWidget="Spinner";
float UIMin=0.0;
float UIMax=10.0;
> = {1.0};

float ECCDesaturateShadows
<
string UIName="CC: Desaturate shadows";
string UIWidget="Spinner";
float UIMin=0.0;
float UIMax=1.0;
> = {0.0};

float3 ECCColorBalanceShadows <
string UIName="CC: Color balance shadows";
string UIWidget="Color";
> = {0.5, 0.5, 0.5};

float3 ECCColorBalanceHighlights <
string UIName="CC: Color balance highlights";
string UIWidget="Color";
> = {0.5, 0.5, 0.5};

float3 ECCChannelMixerR <
string UIName="CC: Channel mixer R";
string UIWidget="Color";
> = {1.0, 0.0, 0.0};

float3 ECCChannelMixerG <
string UIName="CC: Channel mixer G";
string UIWidget="Color";
> = {0.0, 1.0, 0.0};

float3 ECCChannelMixerB <
string UIName="CC: Channel mixer B";
string UIWidget="Color";
> = {0.0, 0.0, 1.0};
#endif //E_CC_PROCEDURAL



//+++++++++++++++++++++++++++++
//external enb parameters, do not modify
//+++++++++++++++++++++++++++++
//x = generic timer in range 0..1, period of 16777216 ms (4.6 hours), y = average fps, w = frame time elapsed (in seconds)
float4 Timer;
//x = Width, y = 1/Width, z = aspect, w = 1/aspect, aspect is Width/Height
float4 ScreenSize;
//changes in range 0..1, 0 means full quality, 1 lowest dynamic quality (0.33, 0.66 are limits for quality levels)
float AdaptiveQuality;
//x = current weather index, y = outgoing weather index, z = weather transition, w = time of the day in 24 standart hours. Weather index is value from weather ini file, for example WEATHER002 means index==2, but index==0 means that weather not captured.
float4 Weather;
//x = dawn, y = sunrise, z = day, w = sunset. Interpolators range from 0..1
float4 TimeOfDay1;
//x = dusk, y = night. Interpolators range from 0..1
float4 TimeOfDay2;
//changes in range 0..1, 0 means that night time, 1 - day time
float ENightDayFactor;
//changes 0 or 1. 0 means that exterior, 1 - interior
float EInteriorFactor;

//+++++++++++++++++++++++++++++
//external enb debugging parameters for shader programmers, do not modify
//+++++++++++++++++++++++++++++
//keyboard controlled temporary variables. Press and hold key 1,2,3...8 together with PageUp or PageDown to modify. By default all set to 1.0
float4 tempF1; //0,1,2,3
float4 tempF2; //5,6,7,8
float4 tempF3; //9,0
// xy = cursor position in range 0..1 of screen;
// z = is shader editor window active;
// w = mouse buttons with values 0..7 as follows:
// 0 = none
// 1 = left
// 2 = right
// 3 = left+right
// 4 = middle
// 5 = left+middle
// 6 = right+middle
// 7 = left+right+middle (or rather cat is sitting on your mouse)
float4 tempInfo1;
// xy = cursor position of previous left mouse button click
// zw = cursor position of previous right mouse button click
float4 tempInfo2;



//+++++++++++++++++++++++++++++
//game and mod parameters, do not modify
//+++++++++++++++++++++++++++++
float4 Params01[6]; //fallout4 parameters
//x - bloom amount; y - lens amount
float4 ENBParams01; //enb parameters

Texture2D TextureColor; //hdr color
Texture2D TextureBloom; //vanilla or enb bloom
Texture2D TextureLens; //enb lens fx
Texture2D TextureDepth; //scene depth
Texture2D TextureAdaptation; //vanilla or enb adaptation
Texture2D TextureAperture; //this frame aperture 1*1 R32F hdr red channel only. computed in depth of field shader file

SamplerState Sampler0
{
Filter = MIN_MAG_MIP_POINT;//MIN_MAG_MIP_LINEAR;
AddressU = Clamp;
AddressV = Clamp;
};
SamplerState Sampler1
{
Filter = MIN_MAG_MIP_LINEAR;
AddressU = Clamp;
AddressV = Clamp;
};



//+++++++++++++++++++++++++++++
//
//+++++++++++++++++++++++++++++
struct VS_INPUT_POST
{
float3 pos : POSITION;
float2 txcoord : TEXCOORD0;
};
struct VS_OUTPUT_POST
{
float4 pos : SV_POSITION;
float2 txcoord0 : TEXCOORD0;
};



//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
VS_OUTPUT_POST VS_Draw(VS_INPUT_POST IN)
{
VS_OUTPUT_POST OUT;
float4 pos;
pos.xyz=IN.pos.xyz;
pos.w=1.0;
OUT.pos=pos;
OUT.txcoord0.xy=IN.txcoord.xy;
return OUT;
}



float4 PS_Draw(VS_OUTPUT_POST IN, float4 v0 : SV_Position0) : SV_Target
{
float4 res;
float4 color;
color=TextureColor.Sample(Sampler0, IN.txcoord0.xy); //hdr scene color

float3 lens;
lens.xyz=TextureLens.Sample(Sampler1, IN.txcoord0.xy).xyz;
color.xyz+=lens.xyz * ENBParams01.y; //lens amount

//TODO add ENBParams01.x as bloom amount to the bloom applied outsize of vanilla post process

#ifdef APPLYGAMECOLORCORRECTION
//fallout4 vanilla post process. Just an example for modders, better not enable without knowing how to edit and what you need
float4 r0, r1, r2, r3;
r0.xyz = color.xyz;
r1.xy = Params01[4].zw * IN.txcoord0.xy;
r1.xyz = TextureBloom.Sample(Sampler1, r1.xy).xyz * ENBParams01.x; //bloom amount
r0.w = TextureAdaptation.Sample(Sampler0, IN.txcoord0.xy).x;
r1.w = Params01[1].z / (0.001 + r0.w);
r2.x = r1.w < Params01[1].y;
r1.w = r2.x ? Params01[1].y : r1.w;
r2.x = Params01[1].x < r1.w;
r1.w = r2.x ? Params01[1].x : r1.w;
r0.xyz = r1.xyz + r0.xyz;
r0.xyz = r0.xyz * r1.w;
r1.xyz = r0.xyz + r0.xyz;
r2.xyz = r0.xyz * 0.3 + 0.05;
r3.xy = float2(0.2, 3.333333) * Params01[1].w;
r2.xyz = r1.xyz * r2.xyz + r3.x;
r0.xyz = r0.xyz * 0.3 + 0.5;
r0.xyz = r1.xyz * r0.xyz + 0.06;
r0.xyz = r2.xyz / r0.xyz;
r0.xyz = -Params01[1].w * 3.333333 + r0.xyz;
r1.x = Params01[1].w * 0.2 + 19.376;
r1.x = r1.x * 0.0408564 - r3.y;
r1.xyz = r0.xyz / r1.x;
r0.x = dot(r1.xyz, float3(0.2125, 0.7154, 0.0721));
r1.xyz = r1.xyz - r0.x;
r1.xyz = Params01[2].x * r1.xyz + r0.x;
r2.xyz = r0.x * Params01[3].xyz - r1.xyz;
r1.xyz = Params01[3].w * r2.xyz + r1.xyz;
r1.xyz = Params01[2].w * r1.xyz - r0.w;
r0.xyz = Params01[2].z * r1.xyz + r0.w;
//last color filter used only for certain conditions, like rifle night scope
color.xyz = lerp(r0.xyz, Params01[5].xyz, Params01[5].w);
color.xyz = saturate(color);
color.xyz = pow(color.xyz, 1.0/2.2);
#endif //APPLYGAMECOLORCORRECTION

{
//float grayadaptation=TextureAdaptation.Sample(Sampler0, IN.txcoord0.xy).x; UNCOMMENT THIS FOR FALLOUT 4
color.rgb=saturate(color.rgb);
float3 brightness=grayadaptation;//adaptation luminance
brightness=(brightness.x/(brightness.x+1.0));//new version
float3 palette;
float4 uvsrc=0.0;
uvsrc.y=brightness.r;
uvsrc.x=color.r;
palette.r=PaletteTexture.Sample(Sampler1, uvsrc).r;
uvsrc.x=color.g;
uvsrc.y=brightness.g;
palette.g=PaletteTexture.Sample(Sampler1, uvsrc).g;
uvsrc.x=color.b;
uvsrc.y=brightness.b;
palette.b=PaletteTexture.Sample(Sampler1, uvsrc).b;
color.rgb=palette.rgb;
}
#ifdef E_CC_PROCEDURAL
//activated by UseProceduralCorrection=true
float tempgray;
float4 tempvar;
float3 tempcolor;

//+++ levels like in photoshop, including gamma, lightness, additive brightness
color=max(color-ECCInBlack, 0.0) / max(ECCInWhite-ECCInBlack, 0.0001);
if (ECCGamma!=1.0) color=pow(color, ECCGamma);
color=color*(ECCOutWhite-ECCOutBlack) + ECCOutBlack;

//+++ brightness
color=color*ECCBrightness;

//+++ contrast
color=(color-ECCContrastGrayLevel) * ECCContrast + ECCContrastGrayLevel;

//+++ saturation
tempgray=dot(color, 0.3333);
color=lerp(tempgray, color, ECCSaturation);

//+++ desaturate shadows
tempgray=dot(color, 0.3333);
tempvar.x=saturate(1.0-tempgray);
tempvar.x*=tempvar.x;
tempvar.x*=tempvar.x;
color=lerp(color, tempgray, ECCDesaturateShadows*tempvar.x);

//+++ color balance
color=saturate(color);
tempgray=dot(color, 0.3333);
float2 shadow_highlight=float2(1.0-tempgray, tempgray);
shadow_highlight*=shadow_highlight;
color.rgb+=(ECCColorBalanceHighlights*2.0-1.0)*color * shadow_highlight.x;
color.rgb+=(ECCColorBalanceShadows*2.0-1.0)*(1.0-color) * shadow_highlight.y;

//+++ channel mixer
tempcolor=color;
color.r=dot(tempcolor, ECCChannelMixerR);
color.g=dot(tempcolor, ECCChannelMixerG);
color.b=dot(tempcolor, ECCChannelMixerB);
#endif //E_CC_PROCEDURAL


res.xyz=saturate(color);
res.w=1.0;
return res;
}



//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//Vanilla post process. Do not modify
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
float4 PS_DrawOriginal(VS_OUTPUT_POST IN, float4 v0 : SV_Position0) : SV_Target
{
float4 res;
float4 color;
color=TextureColor.Sample(Sampler0, IN.txcoord0.xy); //hdr scene color

float4 r0, r1, r2, r3;
r0.xyz = color.xyz;
r1.xy = Params01[4].zw * IN.txcoord0.xy;
r1.xyz = TextureBloom.Sample(Sampler1, r1.xy).xyz;
r0.w = TextureAdaptation.Sample(Sampler0, IN.txcoord0.xy).x;
r1.w = Params01[1].z / (0.001 + r0.w);
r2.x = r1.w < Params01[1].y;
r1.w = r2.x ? Params01[1].y : r1.w;
r2.x = Params01[1].x < r1.w;
r1.w = r2.x ? Params01[1].x : r1.w;
r0.xyz = r1.xyz + r0.xyz;
r0.xyz = r0.xyz * r1.w;
r1.xyz = r0.xyz + r0.xyz;
r2.xyz = r0.xyz * 0.3 + 0.05;
r3.xy = float2(0.2, 3.333333) * Params01[1].w;
r2.xyz = r1.xyz * r2.xyz + r3.x;
r0.xyz = r0.xyz * 0.3 + 0.5;
r0.xyz = r1.xyz * r0.xyz + 0.06;
r0.xyz = r2.xyz / r0.xyz;
r0.xyz = -Params01[1].w * 3.333333 + r0.xyz;
r1.x = Params01[1].w * 0.2 + 19.376;
r1.x = r1.x * 0.0408564 - r3.y;
r1.xyz = r0.xyz / r1.x;
r0.x = dot(r1.xyz, float3(0.2125, 0.7154, 0.0721));
r1.xyz = r1.xyz - r0.x;
r1.xyz = Params01[2].x * r1.xyz + r0.x;
r2.xyz = r0.x * Params01[3].xyz - r1.xyz;
r1.xyz = Params01[3].w * r2.xyz + r1.xyz;
r1.xyz = Params01[2].w * r1.xyz - r0.w;
r0.xyz = Params01[2].z * r1.xyz + r0.w;
//last color filter used only for certain conditions, like rifle night scope
res.xyz = lerp(r0.xyz, Params01[5].xyz, Params01[5].w);

res.xyz = pow(res.xyz, 1.0/2.2);
res.w=1.0;
return res;
}



//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//techniques
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
technique11 Draw <string UIName="ENBSeries";>
{
pass p0
{
SetVertexShader(CompileShader(vs_5_0, VS_Draw()));
SetPixelShader(CompileShader(ps_5_0, PS_Draw()));
}
}



technique11 ORIGINALPOSTPROCESS <string UIName="Vanilla";> //do not modify this technique
{
pass p0
{
SetVertexShader(CompileShader(vs_5_0, VS_Draw()));
SetPixelShader(CompileShader(ps_5_0, PS_DrawOriginal()));
}
}

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17427
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: enb 0.323 for fallout cant use palettes.

I told you to uncomment line, but you uncommented comment, not the line itself.
WRONG:

Code: Select all

//float grayadaptation=TextureAdaptation.Sample(Sampler0, IN.txcoord0.xy).x; UNCOMMENT THIS FOR FALLOUT 4
RIGHT:

Code: Select all

float grayadaptation=TextureAdaptation.Sample(Sampler0, IN.txcoord0.xy).x; //UNCOMMENT THIS FOR FALLOUT 4
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
Posts: 6
Joined: 17 Feb 2018, 05:59

Re: enb 0.323 for fallout cant use palettes.

Thank you so much for the replies this has it working for me.
Post Reply