Jump to content

  • Log In with Google      Sign In   
  • Create Account


Vertex Shader unexpected results, bizarre

  • You cannot reply to this topic
9 replies to this topic

#1 danromeo   Members   -  Reputation: 222

Like
0Likes
Like

Posted 08 May 2014 - 09:09 PM

Hi.

 

I have a version 3.0 vertex shader for (what I call) Static Imposters that essentially decides which image in a texture atlas should be used and sends that info to the pixel shader.  

 

I am confident that the shader is written correctly but it always displays the wrong image!  If I run the shader through a debugger, the numbers are all correct, but the wrong image displays.  I have isolated this to a single variable, whose value is assigned from the program with Effect.Parameters["ImagesPerView"].SetValue(ImagesPerView);  

 

Running the shader through the Pix debugger, the value of ImagesPerView is always set correctly to 12, but the program displays the image as if the value of ImagesPerView is set to 11.  If I hardcode the value of ImagesPerView in the vertex shader to 12, overriding the program assignment, the program displays the correct image!  WHY am I getting different results even though the value of ImagesPerView is always 12?  

 

The program is assigning the value of 12 to ImagesPerView on every draw call.  I don't think this is due to any sort of  implicit conversions as I have checked the code thoroughly, and even did an (int)ImagesPerView conversion on every operation to make sure.  All of the other variable assignments in the shader appear to be correct and functioning correctly.  Stared at it until I was crosseyed.....What The Heck am I doing wrong?  

Any ideas welcome.

 

THANKS



Sponsor:

#2 newtechnology   Members   -  Reputation: 583

Like
1Likes
Like

Posted 08 May 2014 - 11:27 PM

Maybe this could be because of wrong indexing?

For instance, there is an array of size 4.

int array[4]

And you are trying to access last element using array[4] instead of array[3].

#3 Hodgman   Moderators   -  Reputation: 27814

Like
3Likes
Like

Posted 09 May 2014 - 12:29 AM

Maybe post some shader code? How is ImagesPerView used in the shader?



#4 Buckeye   Crossbones+   -  Reputation: 3808

Like
2Likes
Like

Posted 09 May 2014 - 03:12 AM

As Hodgman mentions, you'll have to post some code.

 


..the value of ImagesPerView is always set correctly to 12, ... If I hardcode the value of ImagesPerView in the vertex shader to 12, overriding the program assignment, the program displays the correct image!

 

That is likely the best indication of the area where the error is taking place.

 

When you post the code, indicate exactly where in the shader code you override the shader value to "12." Also, post some code around the place in your program where you send the variable to the shader. If you use Effect.Parameters["ImagesPerView"].SetValue(12) [ I don't mean ..SetValue(ImagesPerView) ] and it doesn't work, but you hardcode ImagesPerView = 12 in the shader, there's (obviously) a problem there.

 

You might also try adding another variable to your shader which you hardcode to 12, and use that value for the calc. Don't change anything else in either the shader or your program.


Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#5 danromeo   Members   -  Reputation: 222

Like
0Likes
Like

Posted 09 May 2014 - 10:59 AM

Indexing, definitely a good idea but I've already checked this.  

 

Also if I throw a hard value from the program (Effect.Parameters["ImagesPerView"].SetValue(12)) it still seems to munge the value, but if I hard set a variable value in the shader (ImagesPerView=12) everything displays correctly.  What's the difference??

 

I'm not an expert but I know shaders make some assumptions about variables.  For example in different parts of the program I'm sending different values to ImagesPerView....but I still don't understand how the debugger can display all of the correct values and yet the program seems to be doing something entirely different.  

 

Rebuilt the entire project....still same results.  

 

Before I post code I'm going to remove all of the instances of throwing different values to ImagesPerView and then strip the shader code down to it's bare minimum.  This looks like some sort of munging to me.....for example hard coding the variable value in the shader results in a lot of code being skipped, so the problem could be in any line of the skipped code.  There is a small bit of branching in the shader that I'm suspicious of, although I've used branching in shaders lots of times without any problems.  it stands to reason that if all the vertex shader did was accept a value from the program and send it to the pixel shader that it would send the correct value!  

 

If any of this inspires any ideas please speak up!  THANKS for your suggestions.  



#6 unbird   Crossbones+   -  Reputation: 4847

Like
3Likes
Like

Posted 09 May 2014 - 11:15 AM

Lucky guess: Rounding errors. SM 3 doesn't really do integers and PIX might display e.g. a 11.9999 as 12, while the conversion yields 11.



#7 danromeo   Members   -  Reputation: 222

Like
0Likes
Like

Posted 11 May 2014 - 03:42 PM

I thought for sure that unbird must be right with rounding errors and/or int behavior, but I think I've covered this and I'm still getting incorrect results.  

 

Declaring all globals as floats in the shader.  Rounding values up to the nearest integer and truncating.  Still the same results and still seems to come down to this one single global.  Again, the number I'm sending to the pixel shader displays in PIX as 0.00 but the program behaves as if it is a different value.  

 

More Facts: If I hard set the global value in the .fx file and never throw a value from the program at all I still get the same incorrect results.   BUT if I declare a variable in the vertex shader and assign the same value everything displays correctly.  This must be a significant fact....not sure what to do with it though.  What's the difference between a global and a variable in an .fx file?  How are they treated differently?  

 

I tried changing the declaration order in the .fx file, still same results.  I keep thinking it must be a syntax error somewhere in the .fx file but I can't find any problems and obviously it compiles.  



#8 Buckeye   Crossbones+   -  Reputation: 3808

Like
0Likes
Like

Posted 11 May 2014 - 04:07 PM

You still haven't posted any code. You're asking for help with your code, but, for some reason, you don't want anyone to see it?

 


More Facts: ...

 

Forget the descriptions. If you want more than guesses, post the code for what you're describing.


Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly.


#9 Hodgman   Moderators   -  Reputation: 27814

Like
3Likes
Like

Posted 11 May 2014 - 05:25 PM

Gobal values are given the 'uniform' qualifier by default, which means their values are supplied by the CPU (the Effect system in your case).
On the other hand, if you hard-code a literal value as a local variable, the compiler knows it will never change and will perform algebraic optimization of your code, and branch removal.

Also, there's (almost) no such thing as int in SM3 (they're emulated using floats in most cases), so it's better to use floor/round/etc if you're trying to do that in the shader...

So... Your code, below, returns different results based on whether you pass a or b into 'Something'... But what is inside 'Something'?
uniform float a = 12.0;float4 main() : COLOR{  float b = 12.0;  return Something(...a or b...);}


#10 danromeo   Members   -  Reputation: 222

Like
0Likes
Like

Posted 01 June 2014 - 10:59 AM

This topic is old because I've been involved with other things.  I've seen conflicting info re: use of int's, seems I saw a MS recommendation to use int's to avoid rounding errors.  Regardless, I get better results when replacing ints with floats.   Thanks!







PARTNERS