HLSL question -- passing data between vertex and pixel shaders
Author
I wrote a shader program in HLSL using DX effects that would blend various textures based on their Y position. As far as I can tell, I'm pretty sure my code is correct (hours spent going over my not-too-complex algorithm!). I am debugging it in Visual Studio .NET using effect edit and I see very strange results. This is my first unique shader program so I'm most likely just doing something wrong.
So in my vertex shader, I'm declaring it as follows:
void RenderTerrainVS( float4 Pos : POSITION,
float3 Normal : NORMAL,
float2 Tex : TEXCOORD0,
out float4 oPos : POSITION,
out float4 Diffuse : COLOR,
out float2 oTex : TEXCOORD0,
out float3 oNormal : TEXCOORD1,
out float3 oViewPos : TEXCOORD2,
out float3 oBlendInfo : TEXCOORD3 )
This is copied from an existing shader that basically just lights the scene. My addition is the last out variable, oBlendInfo. I just used the TEXCOORD3 register for the lack of any clue what I should use (and I saw some other shader in some microsoft sample using it and my shader didn't appear to be using it). Is this a reasonable one to use? How do I know how much data can fit in a register? How do I know which ones are available for use and won't clobber data stored for later?
When I set the data in oBlendInfo, I do various calculations and end up with the following code:
// Set the data to be passed on to the pixel shader
oBlendInfo.x = (float)layer; // The layer we are in
oBlendInfo.y = fBlendAmt;
oBlendInfo.z = fLayerSection;
layer is an int that I'm piggy backing on this variable mostly since I don't know what other registers to use for outgoing.
So my pixel shader looks like this:
void RenderTerrainPS( float4 Diffuse : COLOR,
float2 Tex : TEXCOORD0,
float3 Normal : TEXCOORD1,
float3 Pos : TEXCOORD2,
float3 BlendInfo : TEXCOORD3,
out float4 oCol : COLOR0,
out float4 oNormal : COLOR1,
out float4 oPos : COLOR2 )
When BlendInfo comes in here, the members are all wacky. Let's say I pass (in the v shader) x = 3, y = 0.345, z = -10.0. When it comes into the pixel shader, my data now looks like x = 2.99843, y = 0.3411, z = -9.97949. What's up with that? As I step through the function, BlendInfo loses its data somehow (which shouldn't really matter since I already set variables with the data but it concerns me that it's disappearing). When looking at it in the debugger, after I set a variable with BlendInfo.x for example, the .x member is now a *. My guess is that it is an optimization because I'm not using .x anymore so it just now lives in the variable.
Actually, I just had a thought, is it interpolating the values based on the pixel's location from the three vertices because I'm using a TEXCOORD register?
I am loving HLSL and shaders but there are few things that just confuse the hell out of me!
Each register can store a float4. Any of the texcoord registers are available for use. Just make sure that your vertex shader outputs match up with your pixel shader inputs.
Yep, the value will get interpolated across the primitive. If you want it is constant across the primitive, make sure it is the same at all three vertices of the primitive.
Glad you like HLSL :)
xyzzy
Yep, the value will get interpolated across the primitive. If you want it is constant across the primitive, make sure it is the same at all three vertices of the primitive.
Glad you like HLSL :)
xyzzy
Author
Thanks for the reply. But I was hoping I was doing something wrong!! I can't figure out this bug in my shader program. Here's the source for it, maybe some of you experts can figure it out. I've gone through a number of iterations but this seems to be pretty close but not quite right...
Here's a screen shot of the problem. It's supposed to be blending seamlessly between each layer but you can see the stripping that is going on at the border.
Here is a snippet of my shader source.
Here's a screen shot of the problem. It's supposed to be blending seamlessly between each layer but you can see the stripping that is going on at the border.
Here is a snippet of my shader source.
const half g_fNumTerrainLayers = 4.0f; // How many textures are in the texture layers//float g_fTerrainBasePoint = 0.0f; // Where the root of the terrain is (the Y coord of the origin of the root point)float g_fLayerWidth = 1.0f / 4.0f; // How wide each texture layer isfloat g_fBlendSection = 0.25f; // Blend on the upper and lower quarter of the texture layer// First layer -- watertexture g_texLayer1; //< string name = "Media\\Textures\\water.tga"; >;sampler2D texLayer1Sampler = sampler_state{ Texture = <g_texLayer1>; MipFilter = LINEAR; MinFilter = LINEAR; MagFilter = LINEAR;};// Second layer -- sandtexture g_texLayer2; //< string name = "Media\\Textures\\sand.tga"; >;sampler2D texLayer2Sampler = sampler_state{ Texture = <g_texLayer2>; MipFilter = LINEAR; MinFilter = LINEAR; MagFilter = LINEAR;};// Third layer -- grasstexture g_texLayer3; //< string name = "Media\\Textures\\grass2.tga"; >;sampler2D texLayer3Sampler = sampler_state{ Texture = <g_texLayer3>; MipFilter = LINEAR; MinFilter = LINEAR; MagFilter = LINEAR;};// Fourth layer -- rocktexture g_texLayer4; //< string name = "Media\\Textures\\mossy_rock.tga"; >;sampler2D texLayer4Sampler = sampler_state{ Texture = <g_texLayer4>; MipFilter = LINEAR; MinFilter = LINEAR; MagFilter = LINEAR;};//// Function to find the textures to use and the blend amount// There are four layers (water, sand, grass, rock).//float4 TerrainTextureLayer( float3 fBlendInfo, float2 Tex : TEXCOORD0 ) : COLOR{ float4 current = 0.0f; // The current layer's texture float4 other = 0.0f; // The other layer's texture float4 retVal = 0.0f; // The resulting texture color int nLayer = (int)(fBlendInfo.x); // The layer we reside in float fCurrBlendFactor = 1.0f - fBlendInfo.y; // Get the relative blend factor for the current layer. float4 blendVals = float4( fCurrBlendFactor, fCurrBlendFactor, fCurrBlendFactor, fCurrBlendFactor ); float fLayerSection = fBlendInfo.z; // Figure out which layer we reside in and blend between the result // Layer 4 -- Mossy Rock if( nLayer >= 3 ) { // Add the Layer 4 texture current = tex2D( texLayer4Sampler, Tex ); // No upper section if( fLayerSection < 0.0f ) { // Now blend the Layer 3 texture other = tex2D( texLayer3Sampler, Tex ); // Now linearly interpolate between the textures // This linearly interpolates between x and y, such that the return value is x when s is 0, and y when s is 1. // lerp( x, y, s ) Returns x + s(y - x). retVal = lerp( current, other, blendVals ); } else retVal = current; } else if( nLayer >= 2 ) // Layer 3 -- Grass { // Add the Layer 3 texture current = tex2D( texLayer3Sampler, Tex ); // Upper section if( fLayerSection > 0.0f ) { // Layer 4 texture other = tex2D( texLayer4Sampler, Tex ); // Now linearly interpolate between the textures retVal = lerp( current, other, blendVals ); } else if( fLayerSection < 0.0f ) // Lower section { // Layer 2 texture other = tex2D( texLayer2Sampler, Tex ); // Now linearlly interpolate between the textures retVal = lerp( current, other, blendVals ); } else retVal = current; } else if( nLayer >= 1 ) // Layer 2 -- Sand { // Add the Layer 2 texture current = tex2D( texLayer2Sampler, Tex ); // Upper section if( fLayerSection > 0.0f ) { // Layer 3 texture other = tex2D( texLayer3Sampler, Tex ); // Now linearly interpolate between the textures retVal = lerp( current, other, blendVals ); } else if( fLayerSection < 0.0f ) // Lower section { // Layer 1 texture other = tex2D( texLayer1Sampler, Tex ); // Now linearly interpolate between the textures retVal = lerp( current, other, blendVals ); } else retVal = current; } else // Layer 1 -- Water { // Add the texture but darken it as it gets deeper (it's water after all...) // Negative clamps to 0.0f current = tex2D( texLayer1Sampler, Tex ) * fBlendInfo.y; // Darken as it goes down (to show depth) // Upper section (no lower section) if( fLayerSection > 0.0f ) { // Layer 2 texture other = tex2D( texLayer2Sampler, Tex ); // Now linearly interpolate between the textures retVal = lerp( current, other, blendVals ); } else retVal = current; } return retVal;}//--------------------------------------------------------------------------------------// Shaders//--------------------------------------------------------------------------------------// Terrainvoid RenderTerrainVS( float4 Pos : POSITION, float3 Normal : NORMAL, float2 Tex : TEXCOORD0, out float4 oPos : POSITION, out float4 Diffuse : COLOR, out float2 oTex : TEXCOORD0, out float3 oNormal : TEXCOORD1, out float3 oViewPos : TEXCOORD2, out float3 oBlendInfo : TEXCOORD3 ){ // Output screen space position oViewPos = mul( Pos, g_mWorldView ); oPos = mul( float4( oViewPos, 1.0f ), g_mProj ); // Transform normal to view space oNormal = mul( Normal, (float3x3)g_mWorldView ); // Compute diffuse Diffuse = 1.0f;//dot( g_vLightDirView, oNormal ); //Note: uncomment this to use lighting info // Propagate tex coord oTex = Tex; // Get the layer and the blend factor of this layer's texture (from the top) int layer = (int)(Pos.y / g_fLayerWidth);// oBlendInfo.y = max( 1.0f, min( 0.0f, (Pos.y - ((float)layer * g_fLayerWidth)) / g_fLayerWidth ) ); // The blend value relative to the lower layer // Find where in the texture layer we are float fLayerAmt = Pos.y - ((float)layer * g_fLayerWidth); fLayerAmt /= g_fLayerWidth; // Converts to 0.0 - 1.0 range // Get the blend amount of our layer's texture float fLayerSection = 0.0f; // Default in middle float fBlendAmt = 1.0f; if( fLayerAmt < g_fBlendSection ) { fBlendAmt = fLayerAmt / (g_fBlendSection * 2.0f) + 0.5f; fLayerSection = -10.0f; // Lower section } else if( fLayerAmt > g_fBlendSection * 3.0f ) { fBlendAmt = (fLayerAmt - (g_fBlendSection * 2.0f)) / (g_fBlendSection * 2.0f) - 0.5f; fLayerSection = 10.0f; // Upper section } // Set the data to be passed on to the pixel shader oBlendInfo.x = (float)layer; // The layer we are in oBlendInfo.y = fBlendAmt; oBlendInfo.z = fLayerSection;}void RenderTerrainPS( float4 Diffuse : COLOR, float2 Tex : TEXCOORD0, float3 Normal : TEXCOORD1, float3 Pos : TEXCOORD2, float3 BlendInfo : TEXCOORD3, out float4 oCol : COLOR0, out float4 oNormal : COLOR1, out float4 oPos : COLOR2 ){ // // Output pixel color // oCol = TerrainTextureLayer( BlendInfo, Tex ) * Diffuse; // // Output normal // oNormal = float4( normalize( Normal ), 1.0f ); // // Output view position // oPos = float4( Pos, 1.0f );}
I haven't looked at your code, but just so you know, the Microsoft HLSL compiler DOES have some bugs in it. If you're sure the code works, try rearranging the code and see if that helps.
To better iterate this - Microsoft's HLSL compiler is known to cause problems with certain sequences of commands. If you're having problems, and you know it should work, try reordering your code to see if it helps.
Jim
Jim
You should ask this on the DX beta newsgroups, just to make sure it's not a beta bug. I'm assuming that's what you're using from the UI components, ignore me if I'm wrong.
They already know about it (I'm on the beta team). I have some code that won't compile correctly on any of the HLSL compilers to date (a, b, c, and rc0).
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
