tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

about everything
  • Author
  • Message
Offline
Posts: 18
Joined: 19 Dec 2022, 15:05

tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

Hello, dear ENBseries and others. How to port tex2Dlod line to dx11 shader? Whats the syntax? Here is example: tex2Dlod(SamplerNoise, coord), i want the same value but in dx11 syntax. Please, help me

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17447
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

It's turns to something like TextureNoise.SampleLevel(SamplerNoise, coord.xy, coord.w); //coord.w usually can be replaced by 0 in external shaders
Of course both noise texture and samplers must exist. And as i wrote in previous topic, they are separate, you can have 16 samplers max, but many more textures. Canuse same sampler for many textures, like:
TextureNoise.SampleLevel(SamplerNoise, coord.xy, coord.w);
TextureColor.SampleLevel(SamplerNoise, coord.xy, coord.w);
TextureDepth.SampleLevel(SamplerNoise, coord.xy, coord.w);
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
Posts: 18
Joined: 19 Dec 2022, 15:05

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

ENBSeries wrote: 03 Jan 2023, 01:54 It's turns to something like TextureNoise.SampleLevel(SamplerNoise, coord.xy, coord.w); //coord.w usually can be replaced by 0 in external shaders
Of course both noise texture and samplers must exist. And as i wrote in previous topic, they are separate, you can have 16 samplers max, but many more textures. Canuse same sampler for many textures, like:
TextureNoise.SampleLevel(SamplerNoise, coord.xy, coord.w);
TextureColor.SampleLevel(SamplerNoise, coord.xy, coord.w);
TextureDepth.SampleLevel(SamplerNoise, coord.xy, coord.w);
How do i need to write textures and Samplers? Should they be like in default dx11 enbseries?

Like this?:

Texture2D TextureOriginal; //color R10B10G10A2 32 bit ldr format
Texture2D TextureColor; //color which is output of previous technique (except when drawed to temporary render target), R10B10G10A2 32 bit ldr format
Texture2D TextureDepth; //scene depth R32F 32 bit hdr format

//temporary textures which can be set as render target for techniques via annotations like <string RenderTarget="RenderTargetRGBA32";>
Texture2D RenderTargetRGBA32; //R8G8B8A8 32 bit ldr format
Texture2D RenderTargetRGBA64; //R16B16G16A16 64 bit ldr format
Texture2D RenderTargetRGBA64F; //R16B16G16A16F 64 bit hdr format
Texture2D RenderTargetR16F; //R16F 16 bit hdr format with red channel only
Texture2D RenderTargetR32F; //R32F 32 bit hdr format with red channel only
Texture2D RenderTargetRGB32F; //32 bit hdr format without alpha

SamplerState Sampler0
{
Filter = MIN_MAG_MIP_POINT;//MIN_MAG_MIP_LINEAR;
AddressU = Clamp;
AddressV = Clamp;
};
SamplerState Sampler1
{
Filter = MIN_MAG_MIP_LINEAR;
AddressU = Clamp;
AddressV = Clamp;
};

And how to declarate 'TextureNoise' and other Samplers if there are only Sampler 0 and Sampler 1?

Offline
Posts: 18
Joined: 19 Dec 2022, 15:05

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

ENBSeries wrote: 03 Jan 2023, 01:54 It's turns to something like TextureNoise.SampleLevel(SamplerNoise, coord.xy, coord.w); //coord.w usually can be replaced by 0 in external shaders
Of course both noise texture and samplers must exist. And as i wrote in previous topic, they are separate, you can have 16 samplers max, but many more textures. Canuse same sampler for many textures, like:
TextureNoise.SampleLevel(SamplerNoise, coord.xy, coord.w);
TextureColor.SampleLevel(SamplerNoise, coord.xy, coord.w);
TextureDepth.SampleLevel(SamplerNoise, coord.xy, coord.w);
Please, can you explain me how to write technique to dx11? In default Skyrim SE enbseries's enbeffectpostpass.fx as many technique11 as many functions with names of effects.
Like this:
float4 PS_Sharp ...
float4 PS_Blur ...
...
technique11 Sharp ...
technique11 Blur ...
...
But in LE preset's shader only 1 float4 with the name of effect and only one technique probably for this one only:
float4 PS_PostProcess ...
technique PostProcess

(In float4 PS_PostProccess: "//sharpening, //grain noise, //FFXAA, //vibrancy" parts)

Should i just remove all default techniques11 (i removed all default effects and copy-pasted float4 PS_PostProccess instead of them) and paste instead of them only one technique from dx9 code, and just add '11' to 'technique'?

Thank you

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17447
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

Techniques in old versions of the mod had index in their names to execute one after another to apply multipass effects, for example PostProcess, PostProcess2, PostProcess3, PostProcess4 means their execution order. After some of version this improved and allow to select which of such possible to use, for example PostProcess, PostProcess2, PostProcess4, Blur, Blur2, Blur3, Blur4 means that in the shader editor window you can select technique PostProcess or Blur and they will switch shaders, but they both gonna execute all those 4 times. Technique name is what define what shader executes. If you see dx9 technique and want to port it to dx11, then just do basic changes like following:
dx9:
technique SomeTechniqueName
{
pass P0
{

VertexShader = compile vs_3_0 VS_SomeShaderName();
PixelShader = compile ps_3_0 PS_SomeShaderName();

DitherEnable=FALSE;
ZEnable=FALSE;
CullMode=NONE;
ALPHATESTENABLE=FALSE;
SEPARATEALPHABLENDENABLE=FALSE;
AlphaBlendEnable=FALSE;
StencilEnable=FALSE;
FogEnable=FALSE;
SRGBWRITEENABLE=FALSE;
}
}
dx11:
technique11 SomeTechniqueName
{
pass p0
{
SetVertexShader(CompileShader(vs_5_0, VS_SomeShaderName()));
SetPixelShader(CompileShader(ps_5_0, PS_SomeShaderName()));
}
}

If you see extra text, like technique Sharp <string UIName="Sharp";> then it's annotation and you should keep it, making technique11 Sharp <string UIName="Sharp";> in dx11.
In general, you need to use application which compares files, i use BeyondCompare 3 for this purpose and when take two files to compare side by side it's way simpler to understand what difference is between dx9 and dx11, all you need is default shaders then, may not need from skyrim, but from other games i modified to make them as close as possible to each other to let you understand first.

Textures globally remain the same, dont touch dx11 ones like TextureColor f.e. But you can add other textures, which were made by preset maker, for LUT or some other on screen effects. Sampler for noise would be something like the following:
SamplerState Sampler2
{
Filter = MIN_MAG_MIP_POINT;
AddressU = Wrap;
AddressV = Wrap;
};
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
Posts: 18
Joined: 19 Dec 2022, 15:05

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

ENBSeries wrote: 03 Jan 2023, 12:58 Techniques in old versions of the mod had index in their names to execute one after another to apply multipass effects, for example PostProcess, PostProcess2, PostProcess3, PostProcess4 means their execution order. After some of version this improved and allow to select which of such possible to use, for example PostProcess, PostProcess2, PostProcess4, Blur, Blur2, Blur3, Blur4 means that in the shader editor window you can select technique PostProcess or Blur and they will switch shaders, but they both gonna execute all those 4 times. Technique name is what define what shader executes. If you see dx9 technique and want to port it to dx11, then just do basic changes like following:
dx9:
technique SomeTechniqueName
{
pass P0
{

VertexShader = compile vs_3_0 VS_SomeShaderName();
PixelShader = compile ps_3_0 PS_SomeShaderName();

DitherEnable=FALSE;
ZEnable=FALSE;
CullMode=NONE;
ALPHATESTENABLE=FALSE;
SEPARATEALPHABLENDENABLE=FALSE;
AlphaBlendEnable=FALSE;
StencilEnable=FALSE;
FogEnable=FALSE;
SRGBWRITEENABLE=FALSE;
}
}
dx11:
technique11 SomeTechniqueName
{
pass p0
{
SetVertexShader(CompileShader(vs_5_0, VS_SomeShaderName()));
SetPixelShader(CompileShader(ps_5_0, PS_SomeShaderName()));
}
}

If you see extra text, like technique Sharp <string UIName="Sharp";> then it's annotation and you should keep it, making technique11 Sharp <string UIName="Sharp";> in dx11.
In general, you need to use application which compares files, i use BeyondCompare 3 for this purpose and when take two files to compare side by side it's way simpler to understand what difference is between dx9 and dx11, all you need is default shaders then, may not need from skyrim, but from other games i modified to make them as close as possible to each other to let you understand first.

Textures globally remain the same, dont touch dx11 ones like TextureColor f.e. But you can add other textures, which were made by preset maker, for LUT or some other on screen effects. Sampler for noise would be something like the following:
SamplerState Sampler2
{
Filter = MIN_MAG_MIP_POINT;
AddressU = Wrap;
AddressV = Wrap;
};
Ok, thank you, i did the same and it works.
But compiller makes this error:
invalid subscript 'txcoord'
It says that error happens after Vertex shader, there are some lines about it:

float4 PS_PostProcess(VS_OUTPUT_POST IN, float4 v0 : SV_Position0) : SV_Target
{
float4 res;
float4 coord=0.0;

coord.xy=IN.txcoord.xy; Thats the one
float4 origcolor;

coord.w=0.0;

origcolor=TextureColor.SampleLevel(SamplerColor, coord.xy, coord.w);

I changed this to "coord.xy=IN.txcoord0.xy;"(just added '0' after 'txcoord') and compiller shows no errors.
There was no error about it before i changed 'Data Structure' and 'Vertex Shader' to SE version, but it was imposible for DxEffect to compile this(error- 'no valid VertexShader-PixelShader combination could be found in Technique part: Pass P0' i fixed this one by changes below):
Data Structure Before:

struct VS_OUTPUT_POST {
float4 vpos : POSITION;
float2 txcoord : TEXCOORD0;
};

struct VS_INPUT_POST {
float3 pos : POSITION;
float2 txcoord : TEXCOORD0;
};
Data Structure After:

struct VS_INPUT_POST
{
float3 pos : POSITION;
float2 txcoord : TEXCOORD0;
};
struct VS_OUTPUT_POST
{
float4 pos : SV_POSITION;
float2 txcoord0 : TEXCOORD0;
};

Vertex Shader Before:

VS_OUTPUT_POST VS_PostProcess(VS_INPUT_POST IN)
{
VS_OUTPUT_POST OUT;

float4 pos=float4(IN.pos.x,IN.pos.y,IN.pos.z,1.0);

OUT.vpos=pos;
OUT.txcoord.xy=IN.txcoord.xy;

return OUT;
}
Vertex Shader After:

VS_OUTPUT_POST VS_PostProcess(VS_INPUT_POST IN)
{
VS_OUTPUT_POST OUT;
float4 pos;
pos.xyz=IN.pos.xyz;
pos.w=1.0;
OUT.pos=pos;
OUT.txcoord0.xy=IN.txcoord.xy;
return OUT;
}
As i've written i fixed 'txcoord' error, and compiller shows nothing.
Did i do anything wrong by adding '0' to 'txcoord'?

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17447
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

Adding 0 is nothing special, just changed name of variable. But for porting code probably simpler would be to rename float2 txcoord0 : TEXCOORD0; to float2 txcoord : TEXCOORD0; (or vice versa). Compiler may not bring errors sometime, but shader not work, it's some weird bug by M$ which happens from time to time, so rely first of all on what you see on the display.
Error "no valid VertexShader-PixelShader combination..." can have different meanings, sometime not what you expect at all. You can try to disable pixel shader first or on the contrary, vertex one, then see if no errors by compiler (but visual errors of course gonna happen), then can activate back. Like the following code:
Both enabled:
SetVertexShader(CompileShader(vs_5_0, VS_SomeShaderName()));
SetPixelShader(CompileShader(ps_5_0, PS_SomeShaderName()));
Only vertex shader enabled:
SetVertexShader(CompileShader(vs_5_0, VS_SomeShaderName()));
//SetPixelShader(CompileShader(ps_5_0, PS_SomeShaderName()));
This way can be sure that only one shader function have problem, not both fails to connect each other.
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
Posts: 18
Joined: 19 Dec 2022, 15:05

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

ENBSeries wrote: 03 Jan 2023, 15:22 Adding 0 is nothing special, just changed name of variable. But for porting code probably simpler would be to rename float2 txcoord0 : TEXCOORD0; to float2 txcoord : TEXCOORD0; (or vice versa). Compiler may not bring errors sometime, but shader not work, it's some weird bug by M$ which happens from time to time, so rely first of all on what you see on the display.
Error "no valid VertexShader-PixelShader combination..." can have different meanings, sometime not what you expect at all. You can try to disable pixel shader first or on the contrary, vertex one, then see if no errors by compiler (but visual errors of course gonna happen), then can activate back. Like the following code:
Both enabled:
SetVertexShader(CompileShader(vs_5_0, VS_SomeShaderName()));
SetPixelShader(CompileShader(ps_5_0, PS_SomeShaderName()));
Only vertex shader enabled:
SetVertexShader(CompileShader(vs_5_0, VS_SomeShaderName()));
//SetPixelShader(CompileShader(ps_5_0, PS_SomeShaderName()));
This way can be sure that only one shader function have problem, not both fails to connect each other.

Hello, i was looking at Skyrim LE enbpostpass and Skyrim SE enbpostpass shaders, and i have seen that all postpass shaders on LE enbseries has - ''texture2d texNoise", but SE doesnt.
Is it possible to add this one to Skyrim SE postpass.fx? And if it is, where can i find this texture source? And, are shaders related to each other?
I've made so much attempts, to port postpass.fx, but it doesnt work. It doesnt make any errors but doesnt work in game(
I think that the problem is in Textures-Samplers, but anyway it can be wherever. Enbseries menu works, and shows postpass.fx ui. And i dont know how, but my edited postpass without any default effects, looks similar as default shader.
What do you think about it?

Offline
User avatar
*blah-blah-blah maniac*
Posts: 17447
Joined: 27 Dec 2011, 08:53
Location: Rather not to say

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

Noise texture is obsolette thing from very old version, maybe 2008 or something like that, so i don't keep it in dx11 mod. If code which you are using do need noise texture for something, just make new texture in the shader and load it from the file. Something like following:
Texture2D ExampleTexture
<
string UIName = "Example texture";
string ResourceName = "noise.bmp";
>;
SamplerState ExampleSampler
{
Filter = MIN_MAG_MIP_POINT; //MIN_MAG_MIP_LINEAR //filter type depends from which way to use noise texture
AddressU = Wrap;
AddressV = Wrap;
};

If no compiler errors but don't see anything, then in the end of shader function write some code to visualize that shader code do work at all, for example write red color as output. If can't see it, then something wrong with techniques, their names, order. Or matching vertex and pixel shaders.
_________________
i9-9900k, 64Gb RAM, RTX 3060 12Gb, Win7

Offline
Posts: 18
Joined: 19 Dec 2022, 15:05

Re: tex2Dlod In dx11 postpass.fx shader enbseries Skyrim SE

ENBSeries wrote: 06 Jan 2023, 00:07 Noise texture is obsolette thing from very old version, maybe 2008 or something like that, so i don't keep it in dx11 mod. If code which you are using do need noise texture for something, just make new texture in the shader and load it from the file. Something like following:
Texture2D ExampleTexture
<
string UIName = "Example texture";
string ResourceName = "noise.bmp";
>;
SamplerState ExampleSampler
{
Filter = MIN_MAG_MIP_POINT; //MIN_MAG_MIP_LINEAR //filter type depends from which way to use noise texture
AddressU = Wrap;
AddressV = Wrap;
};

If no compiler errors but don't see anything, then in the end of shader function write some code to visualize that shader code do work at all, for example write red color as output. If can't see it, then something wrong with techniques, their names, order. Or matching vertex and pixel shaders.
Thank you so much for your reply. Where can i find 'noise.bmp' file?
Is this color code right?:
float4 color = TextureColor.Sample(Sampler0, IN.txcoord.xy);
float Exposure = 10.00;
float RedExposure = 2.0;
float GreenExposure = 2.0;
float BlueExposure = 2.0;
color.y *= RedExposure;
color.z *= GreenExposure;
return color;
Post Reply