Jump to content
  • Advertisement
Sign in to follow this  
devronious

Please Help, Weird Pixel Shader Results...

This topic is 4306 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hey there, I'm doing a conditional operation that if true makes the output alpha of my pixel shader 0.5. And if false 1.0. I'm getting alpha where there shouldn't be so I used pix to debug the problem area. There is only one primitive drawn in the area. PIX (and visually inspected) says (in color values (a,r,g,b)): Render Target Clear (1,0,0,0) Pixel Shader Output (0.502,0.424,0.933,0) Final Frame Buffer (0.749,0.212,0.467,0) When I inspect the shader I see that it actually outputs an alpha value of 1 (which is what it should be). So why is it outputting a completely different value? I can't figure this out, is something wrong. Please help, Devin

Share this post


Link to post
Share on other sites
Advertisement
How pixel shader output alpha applies to the framebuffer is hugely dependent on how the device's render state is configured.

That said, if PIX says the pixel shader output has .5 for its alpha component, why do you think it's actually outputting 1? The reason this seems relevant is that the final frame buffer values seem to be about half of the color component outputs from the pixel shader, which would be about right if doing standard alpha blending.

One way of visually debugging this sort of thing is to force the alpha value to 1, and move the computed alpha into rgb - that way your pixels get written with the computed alpha values, letting you quickly see what the results are for all the pixels you're writing.

Share this post


Link to post
Share on other sites
In pix you can see the register values and step thru a pixel shader for any given rendered pixel in a frame. As I stepped thru I verified that the operations were being performed correctly. Each step verifying the register values and operations. It also tells you the output register values. The output alpha was 1.0.

But it also shows you what the final pixel shader output is in a color with rgba values below it, which was 0.5. This makes no sense because they should be the same shouldn't they?

Thanks for your help,

Devin

Share this post


Link to post
Share on other sites
Quote:
Original post by devronious
But it also shows you what the final pixel shader output is in a color with rgba values below it, which was 0.5. This makes no sense because they should be the same shouldn't they?
There is another stage in the pipeline between the PS output and the actual value in the frame buffer - fixed function blending (now known as the "Output Merger" in D3D10). Depending on how your pipeline is configured it can and will alter the values as written to the RT and eventually displayed on screen - this is what JasonBlochowiak is hinting at [smile]

Use PIX to verify your pipeline configuration at the time your Draw call was submitted. You may think you're setting things up correctly, but its all too easy to let an erronous render state slip through the net and get these sorts of problems.

Blending states, write masks, I/O formats etc...

hth
Jack

Share this post


Link to post
Share on other sites
Thanks I wasn't aware, but after you guys brought up this I can see that it was alpha blending. I also found the problem, which was that as I passed the clip data thru the color registers from the vs to the ps it was rounding the float to a byte value and was throwing me off.

When I switched my ps version to 3.0 from 2.0 it was fixed. Is there a way to allow more significance to go thru the color registers of ps2.0?

Thanks,

Devin

Share this post


Link to post
Share on other sites
To my knowledge, the 2 color registers of a ShaderModel 2 pipeline are still low precision (10 or 12 bits) and clamped to a range of (0, 1). You'll have to use one of the texture coord registers instead if you need the full floating point range and precision.

Share this post


Link to post
Share on other sites
Thanks,

If I use the texture coordinate won't it interpolate and change per pixel? I need to keep it to one value for all pixels.

-Devin

Share this post


Link to post
Share on other sites
Schrompf,

Ended up that I could use the Texture Coordinates after all.

Thanks guys, your big help!

tok_junior,

I'm having trouble figuring out how to put data into a fixed register. Perhaps I have to do that using asm lang?

-Devin

Share this post


Link to post
Share on other sites
Quote:
Original post by devronious
tok_junior,

I'm having trouble figuring out how to put data into a fixed register. Perhaps I have to do that using asm lang?
Maybe tok_junior will clarify, but my interpretation of their post is that they've misunderstood your problem - using a CB won't help you here.

hth
Jack

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!