Please Help, Weird Pixel Shader Results...
Author
Hey there,
I'm doing a conditional operation that if true makes the output alpha of my pixel shader 0.5. And if false 1.0.
I'm getting alpha where there shouldn't be so I used pix to debug the problem area. There is only one primitive drawn in the area.
PIX (and visually inspected) says (in color values (a,r,g,b)):
Render Target Clear (1,0,0,0)
Pixel Shader Output (0.502,0.424,0.933,0)
Final Frame Buffer (0.749,0.212,0.467,0)
When I inspect the shader I see that it actually outputs an alpha value of 1 (which is what it should be). So why is it outputting a completely different value?
I can't figure this out, is something wrong.
Please help,
Devin
How pixel shader output alpha applies to the framebuffer is hugely dependent on how the device's render state is configured.
That said, if PIX says the pixel shader output has .5 for its alpha component, why do you think it's actually outputting 1? The reason this seems relevant is that the final frame buffer values seem to be about half of the color component outputs from the pixel shader, which would be about right if doing standard alpha blending.
One way of visually debugging this sort of thing is to force the alpha value to 1, and move the computed alpha into rgb - that way your pixels get written with the computed alpha values, letting you quickly see what the results are for all the pixels you're writing.
That said, if PIX says the pixel shader output has .5 for its alpha component, why do you think it's actually outputting 1? The reason this seems relevant is that the final frame buffer values seem to be about half of the color component outputs from the pixel shader, which would be about right if doing standard alpha blending.
One way of visually debugging this sort of thing is to force the alpha value to 1, and move the computed alpha into rgb - that way your pixels get written with the computed alpha values, letting you quickly see what the results are for all the pixels you're writing.
Author
In pix you can see the register values and step thru a pixel shader for any given rendered pixel in a frame. As I stepped thru I verified that the operations were being performed correctly. Each step verifying the register values and operations. It also tells you the output register values. The output alpha was 1.0.
But it also shows you what the final pixel shader output is in a color with rgba values below it, which was 0.5. This makes no sense because they should be the same shouldn't they?
Thanks for your help,
Devin
But it also shows you what the final pixel shader output is in a color with rgba values below it, which was 0.5. This makes no sense because they should be the same shouldn't they?
Thanks for your help,
Devin
Quote:Original post by devroniousThere is another stage in the pipeline between the PS output and the actual value in the frame buffer - fixed function blending (now known as the "Output Merger" in D3D10). Depending on how your pipeline is configured it can and will alter the values as written to the RT and eventually displayed on screen - this is what JasonBlochowiak is hinting at [smile]
But it also shows you what the final pixel shader output is in a color with rgba values below it, which was 0.5. This makes no sense because they should be the same shouldn't they?
Use PIX to verify your pipeline configuration at the time your Draw call was submitted. You may think you're setting things up correctly, but its all too easy to let an erronous render state slip through the net and get these sorts of problems.
Blending states, write masks, I/O formats etc...
hth
Jack
Author
Thanks I wasn't aware, but after you guys brought up this I can see that it was alpha blending. I also found the problem, which was that as I passed the clip data thru the color registers from the vs to the ps it was rounding the float to a byte value and was throwing me off.
When I switched my ps version to 3.0 from 2.0 it was fixed. Is there a way to allow more significance to go thru the color registers of ps2.0?
Thanks,
Devin
When I switched my ps version to 3.0 from 2.0 it was fixed. Is there a way to allow more significance to go thru the color registers of ps2.0?
Thanks,
Devin
To my knowledge, the 2 color registers of a ShaderModel 2 pipeline are still low precision (10 or 12 bits) and clamped to a range of (0, 1). You'll have to use one of the texture coord registers instead if you need the full floating point range and precision.
Author
Thanks,
If I use the texture coordinate won't it interpolate and change per pixel? I need to keep it to one value for all pixels.
-Devin
If I use the texture coordinate won't it interpolate and change per pixel? I need to keep it to one value for all pixels.
-Devin
Both the texture coordinates and colors are interpolators. If you need a constant value, use a constant register.
Author
Schrompf,
Ended up that I could use the Texture Coordinates after all.
Thanks guys, your big help!
tok_junior,
I'm having trouble figuring out how to put data into a fixed register. Perhaps I have to do that using asm lang?
-Devin
Ended up that I could use the Texture Coordinates after all.
Thanks guys, your big help!
tok_junior,
I'm having trouble figuring out how to put data into a fixed register. Perhaps I have to do that using asm lang?
-Devin
Quote:Original post by devroniousMaybe tok_junior will clarify, but my interpretation of their post is that they've misunderstood your problem - using a CB won't help you here.
tok_junior,
I'm having trouble figuring out how to put data into a fixed register. Perhaps I have to do that using asm lang?
hth
Jack
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
