# hlsl - using camPos to get fog

## Recommended Posts

Hi

Im trying to get fog to work with my custom shader in irrlicht engine. So in c++ i have this

class MyShaderCallBack : public video::IShaderConstantSetCallBack
{
public:
virtual void OnSetConstants(video::IMaterialRendererServices* services,
s32 userData)
{
video::IVideoDriver* driver = services->getVideoDriver();
// set clip matrix
core::matrix4 worldViewProj;
worldViewProj = driver->getTransform(video::ETS_PROJECTION);
worldViewProj *= driver->getTransform(video::ETS_VIEW);
worldViewProj *= driver->getTransform(video::ETS_WORLD);
else

//send camPos
core::vector3df pos = device->getSceneManager()->getActiveCamera()->getAbsolutePosition();

else

}
};



This should send the position of the cam to the shader. In hlsl i have this at the top of the file

loat3 camPos : register( c0 );


And the pixelshader output function (in hlsl) looks like this

PS_OUTPUT ps_main(in VS_OUTPUT input)
{
float texScale = 1.0;
PS_OUTPUT output = (PS_OUTPUT)0;

vector a = tex2D(AlphaMap, input.alphamap);
vector i = tex2D(TextureOne, mul(input.tex, texScale));
vector j = tex2D(TextureTwo, mul(input.tex, texScale));
vector k = tex2D(TextureThree, mul(input.tex, texScale));

float4 oneminusx = 1.0 - a.x;
float4 oneminusy = 1.0 - a.y;
float4 oneminusz = 1.0 - a.z;

vector l = a.x * i + oneminusx * i;
vector m = a.y * j + oneminusy * l;
vector n = a.z * k + oneminusz * m;

//how to make use of camPos to alter diffuse here?

output.diffuse = n;
return output;
}


It does some splatting for me on the terrain. Now i want to use camPos to make linear fog from real color to some grey starting at 1000 and being full fog at 1300 (lets just say). So i have to modify output.diffuse right? But i dont know how to do that... Plz help in any way you can.

Thanks

Erik

Edited by suliman

##### Share on other sites

Well, since you're using DirectX, there are already fog modes implemented and ready to be used, at least in the fixed function pipeline:

http://msdn.microsoft.com/en-nz/library/windows/desktop/bb173401(v=vs.85).aspx

If you don't have access to those, you can implement them yourself (the formulae are there). Simply compute the distance from your pixel to the camera pos (you should have your pixel's interpolated world coordinates somewhere) and use it in whichever formula you prefer, with sensible parameters (fog end, fog start, and so on).

Then the pixel color blending step is really simple - use the obtained value (between 0 and 1) from the previous calculation, and lerp your pixel's color with the fog color (using the value you got as a linear interpolation coefficient). This will get you the correct color for your pixel. I don't know how to use the diffuse thing, though, sorry.

##### Share on other sites

i dont have access since im doing a custom shader for splatting.

thanks for the link but the interpolation coefficient itself (between 0 and 1) is trivial its the other stuff i dont know how to get (i understand the basics of what fog is, only not how to do it, code-wise). The diffuse is simply the color. If i set it to n*0.5 it gets darkened. If i set it to n*0.0 everything gets black.

1. how do i define the color in hlsl that i need for fog?

2. how do i get the distance between my camPos and the vertex? The code above is all that is in the output function so what is the vertex position?

3. once i have the distance, how do i larp original color with fog color?

Thanks again

Ive really tried a lot with this...

E

##### Share on other sites

noone? Im stuck at this point and i really need fog for my terrain.

##### Share on other sites

1. how do i define the color in hlsl that i need for fog?
2. how do i get the distance between my camPos and the vertex? The code above is all that is in the output function so what is the vertex position?
3. once i have the distance, how do i larp original color with fog color?

#1: Pass it to the shader.
#2: By multiplying the depth by the inverse of the projection matrix or by passing depth from the vertex shader. There are many topics on how to reverse depth from a depth buffer if you take the time to Google it. If you don’t have access to the depth buffer and can’t add custom data to vertex shaders, you can still get the depth by taking the vertex position in the pixel shader and multiplying the Z by the W.
#3: OriginalColor * FogFactor + (1 - FogFactor) * FogColor. Do you know where I got that equation? From the link Bacterius just gave you.

For someone so stuck, you don’t seem to be showing any signs of helping yourself. All of these answers are:
#1: Logic.
#2: A Google search for reversing depth from a depth buffer, or logic in the case of just passing depth values from the vertex shader.

L. Spiro

Edited by L. Spiro

##### Share on other sites

Ok thanks guys. I now have the code below. It finally works!

But i use a static Depth so everything gets the same amount of fog. My camPos is a float3. But how do i get distance between it and the current pixel (which i will use as 'Depth')? I guess old pythagoras in 3d is out friend but i cannot get the coords of the pixel.

Or is the xyz-coordinates not possible to get instead i should use the techique you mention above?

Thanks a lot.

Erik

PS_OUTPUT ps_main(in VS_OUTPUT input)
{
float texScale = 1.0;
PS_OUTPUT output = (PS_OUTPUT)0;

vector a = tex2D(AlphaMap, input.alphamap);
vector i = tex2D(TextureOne, mul(input.tex, texScale));
vector j = tex2D(TextureTwo, mul(input.tex, texScale));
vector k = tex2D(TextureThree, mul(input.tex, texScale));

float4 oneminusx = 1.0 - a.x;
float4 oneminusy = 1.0 - a.y;
float4 oneminusz = 1.0 - a.z;

vector l = a.x * i + oneminusx * i;
vector m = a.y * j + oneminusy * l;
vector n = a.z * k + oneminusz * m;

//how to get real Depth?

float FogStart=500;
float FogEnd=1000;
float Depth=750;
float4 FogColor = float4(0.5f, 0.5f, 0.5f, 1.0f);

float factor = (FogEnd - Depth) / (FogEnd - FogStart);
if(factor<0)
factor=0;
if(factor>1)
factor=1;

output.diffuse = n*factor+(1-factor)*FogColor;
return output;

}

Edited by suliman

##### Share on other sites

This:

if(factor<0)
factor=0;
if(factor>1)
factor=1;


factor = saturate( factor );



or

float factor = saturate( (FogEnd - Depth) / (FogEnd - FogStart) );



But how do i get distance between it and the current pixel (which i will use as 'Depth')?

You could for example transform your vertex in world space in vertex shader:

void VertexMain(...)
{
...
OUT.WorldPos = mul( IN.Position, World );
}


then in pixel shader get the distance from camera:

void PixelMain(...)
{
...
float depth = distance( camPos, IN.WorldPos );
...
}

Edited by belfegor

##### Share on other sites

Hi belfegor

I change so my vs_main looks like (adds just one line, the rest is there from the splatting code)

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;
Output.Position = mul( Input.Position, matViewProjection );
Output.alphamap = Input.alphamap;
Output.tex = Input.tex;
Output.WorldPos = mul( Input.Position, World );

return( Output );
}


But this "breaks" the shader (terrain looses texture entirely, like if i write an syntax error). Do i need to register "World" or something? I backtrack and add WorldPos to my VS_INPUT. This works fine.

struct VS_INPUT
{
float4 Position : POSITION0;
float4 WorldPos : POSITION0;
float2 alphamap : TEXCOORD0;
float2 tex : TEXCOORD1;
};


What doesnt work is when i try to add it to my VS_OUTPUT structure like so:

struct VS_OUTPUT
{
float4 Position : POSITION0;
float4 WorldPos : POSITION0;
float2 alphamap : TEXCOORD0;
float2 tex : TEXCOORD1;
};


This is what breaks the shader. Why? Dont I need it here? so i can set it in my VS_MAIN?

Erik

Edited by suliman

##### Share on other sites

Didn't use Irrlicht for years now. I don't see how could it break the code.

Do you do the same as for "matViewProjection"?

services->setVertexShaderConstant("matViewProjection", worldViewProj.pointer(), 16);


if that is correct syntax on Irrlicht side.

Can you see the debug output from shader? There should be logger in Irrlicht that gather shader error/warning messages.

Edited by belfegor

##### Share on other sites

I added the second line to shadercallback. The first line was already the same. (But i use worldViewProd. for the second line not world.)

class MyShaderCallBack : public video::IShaderConstantSetCallBack
{
public:
virtual void OnSetConstants(video::IMaterialRendererServices* services,
s32 userData)
{
video::IVideoDriver* driver = services->getVideoDriver();
// set clip matrix
core::matrix4 worldViewProj;
worldViewProj = driver->getTransform(video::ETS_PROJECTION);
worldViewProj *= driver->getTransform(video::ETS_VIEW);
worldViewProj *= driver->getTransform(video::ETS_WORLD);
}
else

//send camPos
core::vector3df pos = device->getSceneManager()->getActiveCamera()->getAbsolutePosition();

else
}
};


The complete shader is below. Note, the three lines are commented out (now the shader "works" but with same fogdensity for all points). Uncommenting the declaration in struct VS_OUTPUT breaks the shader, and i dont know why...

Thanks again

Erik

float3 camPos : register( c0 );

float4x4 matViewProjection : ViewProjection;

sampler AlphaMap = sampler_state
{
};

sampler TextureOne = sampler_state
{
MipFilter = LINEAR;
MinFilter = LINEAR;
MagFilter = LINEAR;
};

sampler TextureTwo = sampler_state
{
MipFilter = LINEAR;
MinFilter = LINEAR;
MagFilter = LINEAR;
};

sampler TextureThree = sampler_state
{
MipFilter = LINEAR;
MinFilter = LINEAR;
MagFilter = LINEAR;
};

struct VS_INPUT
{
float4 Position : POSITION0;
float4 WorldPos : POSITION0;
float2 alphamap : TEXCOORD0;
float2 tex : TEXCOORD1;
};

struct VS_OUTPUT
{
float4 Position : POSITION0;
//float4 WorldPos : POSITION0;
float2 alphamap : TEXCOORD0;
float2 tex : TEXCOORD1;
};

struct PS_OUTPUT
{
float4 diffuse : COLOR0;
};

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;
Output.Position = mul( Input.Position, matViewProjection );
Output.alphamap = Input.alphamap;
Output.tex = Input.tex;
//Output.WorldPos = mul( Input.Position, World );

return( Output );
}

PS_OUTPUT ps_main(in VS_OUTPUT input)
{
float texScale = 1.0;
PS_OUTPUT output = (PS_OUTPUT)0;

vector a = tex2D(AlphaMap, input.alphamap);
vector i = tex2D(TextureOne, mul(input.tex, texScale));
vector j = tex2D(TextureTwo, mul(input.tex, texScale));
vector k = tex2D(TextureThree, mul(input.tex, texScale));

float4 oneminusx = 1.0 - a.x;
float4 oneminusy = 1.0 - a.y;
float4 oneminusz = 1.0 - a.z;

vector l = a.x * i + oneminusx * i;
vector m = a.y * j + oneminusy * l;
vector n = a.z * k + oneminusz * m;

//we want the real Depth here, not the static one

float FogStart=500;
float FogEnd=1000;
float Depth=750;
//float Depth = distance( camPos, input.WorldPos );
float4 FogColor = float4(0.5f, 0.5f, 0.5f, 1.0f);

//factor
float factor = (FogEnd - Depth) / (FogEnd - FogStart);
factor=saturate(factor);

output.diffuse = n*factor+(1-factor)*FogColor;
return output;

}

technique Default_DirectX_Effect
{
pass Pass_0
{
}
}


Edited by suliman

##### Share on other sites

You didn't supplied correct matrix on Irrlicht side and didn't define in shader. Put these changes:

...
core::matrix4 worldViewProj;
core::matrix4 world;
worldViewProj = driver->getTransform(video::ETS_PROJECTION);
worldViewProj *= driver->getTransform(video::ETS_VIEW);
worldViewProj *= driver->getTransform(video::ETS_WORLD);
world = driver->getTransform(video::ETS_WORLD);
....


float4x4 matViewProjection : register( c0 );
float4x4 World : register( c4 );
float3 camPos : register( c8 );



// This should be your mesh vertex elements
// so this is questionable
struct VS_INPUT
{
float4 Position : POSITION0;
float2 alphamap : TEXCOORD0;
float2 tex : TEXCOORD1;
};

struct VS_OUTPUT
{
float4 Position : POSITION0;
float4 WorldPos : TEXCOORD0; // you cannot have 2 POSITION0!!!
float2 alphamap : TEXCOORD1;
float2 tex : TEXCOORD2;
};



PS_OUTPUT ps_main(in VS_OUTPUT input)
{
...
float Depth = distance( camPos, input.WorldPos.xyz );
...
}


and uncomment relevant stuff in shader

Edited by belfegor

##### Share on other sites

Hmm i do one change at a time and see when the shader breaks:

If i put

float4x4 matViewProjection : register( c0 );
float4x4 World : register( c4 );
float3 camPos : register( c8 );

it breaks. If i comment the first line away it works again. Same thing with changing VS_OUTPUT the way you suggests above; it breaks the shader. Here, if i comment out the WorldPos line the shader works again...

ruct VS_OUTPUT
{
float4 Position : POSITION0;
float4 WorldPos : TEXCOORD0; // unless i comment this line out the shader breaks. Even though it is not used anywhere in the code
float2 alphamap : TEXCOORD1;
float2 tex : TEXCOORD2;
};

Edited by suliman

##### Share on other sites

You need to see debug output to be able to tell what is wrong. What vertex elements your terrain mesh have?

Could you possibly send me your whole source with media, so i can resolve this in 5 minutes instead of guessing couple of days now?

##### Share on other sites

of course, see the attached file. I still dont find a log with shader errors...

##### Share on other sites

Ok. Give me some time...

EDIT: Done.

you missed this on Irrlicht side, even i posted it above

world          = driver->getTransform(video::ETS_WORLD);



Also, you dont need this:

driver->setFog(...)


http://pastebin.com/d22tjM7H

http://pastebin.com/TsQuyC9j

There was debug output in console window!

Now you should play with fog start/end values.

Edited by belfegor

##### Share on other sites

Nice!
It works exactly as intended now. Im very greatful.

On a sidenote, your edited hlsl code results in 800 FPS whereas I had only 150 before. Do you know where this big gain comes from? Did this correct something wasteful i had before in the code?

Again, thanks a lot belfegor

Erik

##### Share on other sites

Do you know where this big gain comes from?


Irrlicht was printing into console (this is slow!) every frame because you had some errors, and that was the reason for slowdown.

##### Share on other sites

ah the irony. Anyway thanks a lot for your help

Erik