Sign in to follow this  

Vertex Shader unexpected results, bizarre

This topic is 1325 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi.

 

I have a version 3.0 vertex shader for (what I call) Static Imposters that essentially decides which image in a texture atlas should be used and sends that info to the pixel shader.  

 

I am confident that the shader is written correctly but it always displays the wrong image!  If I run the shader through a debugger, the numbers are all correct, but the wrong image displays.  I have isolated this to a single variable, whose value is assigned from the program with Effect.Parameters["ImagesPerView"].SetValue(ImagesPerView);  

 

Running the shader through the Pix debugger, the value of ImagesPerView is always set correctly to 12, but the program displays the image as if the value of ImagesPerView is set to 11.  If I hardcode the value of ImagesPerView in the vertex shader to 12, overriding the program assignment, the program displays the correct image!  WHY am I getting different results even though the value of ImagesPerView is always 12?  

 

The program is assigning the value of 12 to ImagesPerView on every draw call.  I don't think this is due to any sort of  implicit conversions as I have checked the code thoroughly, and even did an (int)ImagesPerView conversion on every operation to make sure.  All of the other variable assignments in the shader appear to be correct and functioning correctly.  Stared at it until I was crosseyed.....What The Heck am I doing wrong?  

Any ideas welcome.

 

THANKS

Share this post


Link to post
Share on other sites
Maybe this could be because of wrong indexing?

For instance, there is an array of size 4.

int array[4]

And you are trying to access last element using array[4] instead of array[3].

Share this post


Link to post
Share on other sites

As Hodgman mentions, you'll have to post some code.

 


..the value of ImagesPerView is always set correctly to 12, ... If I hardcode the value of ImagesPerView in the vertex shader to 12, overriding the program assignment, the program displays the correct image!

 

That is likely the best indication of the area where the error is taking place.

 

When you post the code, indicate exactly where in the shader code you override the shader value to "12." Also, post some code around the place in your program where you send the variable to the shader. If you use Effect.Parameters["ImagesPerView"].SetValue(12) [ I don't mean ..SetValue(ImagesPerView) ] and it doesn't work, but you hardcode ImagesPerView = 12 in the shader, there's (obviously) a problem there.

 

You might also try adding another variable to your shader which you hardcode to 12, and use that value for the calc. Don't change anything else in either the shader or your program.

Share this post


Link to post
Share on other sites

Indexing, definitely a good idea but I've already checked this.  

 

Also if I throw a hard value from the program (Effect.Parameters["ImagesPerView"].SetValue(12)) it still seems to munge the value, but if I hard set a variable value in the shader (ImagesPerView=12) everything displays correctly.  What's the difference??

 

I'm not an expert but I know shaders make some assumptions about variables.  For example in different parts of the program I'm sending different values to ImagesPerView....but I still don't understand how the debugger can display all of the correct values and yet the program seems to be doing something entirely different.  

 

Rebuilt the entire project....still same results.  

 

Before I post code I'm going to remove all of the instances of throwing different values to ImagesPerView and then strip the shader code down to it's bare minimum.  This looks like some sort of munging to me.....for example hard coding the variable value in the shader results in a lot of code being skipped, so the problem could be in any line of the skipped code.  There is a small bit of branching in the shader that I'm suspicious of, although I've used branching in shaders lots of times without any problems.  it stands to reason that if all the vertex shader did was accept a value from the program and send it to the pixel shader that it would send the correct value!  

 

If any of this inspires any ideas please speak up!  THANKS for your suggestions.  

Share this post


Link to post
Share on other sites

Lucky guess: Rounding errors. SM 3 doesn't really do integers and PIX might display e.g. a 11.9999 as 12, while the conversion yields 11.

Share this post


Link to post
Share on other sites

I thought for sure that unbird must be right with rounding errors and/or int behavior, but I think I've covered this and I'm still getting incorrect results.  

 

Declaring all globals as floats in the shader.  Rounding values up to the nearest integer and truncating.  Still the same results and still seems to come down to this one single global.  Again, the number I'm sending to the pixel shader displays in PIX as 0.00 but the program behaves as if it is a different value.  

 

More Facts: If I hard set the global value in the .fx file and never throw a value from the program at all I still get the same incorrect results.   BUT if I declare a variable in the vertex shader and assign the same value everything displays correctly.  This must be a significant fact....not sure what to do with it though.  What's the difference between a global and a variable in an .fx file?  How are they treated differently?  

 

I tried changing the declaration order in the .fx file, still same results.  I keep thinking it must be a syntax error somewhere in the .fx file but I can't find any problems and obviously it compiles.  

Share this post


Link to post
Share on other sites

You still haven't posted any code. You're asking for help with your code, but, for some reason, you don't want anyone to see it?

 


More Facts: ...

 

Forget the descriptions. If you want more than guesses, post the code for what you're describing.

Share this post


Link to post
Share on other sites
Gobal values are given the 'uniform' qualifier by default, which means their values are supplied by the CPU (the Effect system in your case).
On the other hand, if you hard-code a literal value as a local variable, the compiler knows it will never change and will perform algebraic optimization of your code, and branch removal.

Also, there's (almost) no such thing as int in SM3 (they're emulated using floats in most cases), so it's better to use floor/round/etc if you're trying to do that in the shader...

So... Your code, below, returns different results based on whether you pass a or b into 'Something'... But what is inside 'Something'?
uniform float a = 12.0;float4 main() : COLOR{  float b = 12.0;  return Something(...a or b...);}

Share this post


Link to post
Share on other sites

This topic is old because I've been involved with other things.  I've seen conflicting info re: use of int's, seems I saw a MS recommendation to use int's to avoid rounding errors.  Regardless, I get better results when replacing ints with floats.   Thanks!

Share this post


Link to post
Share on other sites

This topic is 1325 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this