• Advertisement
Sign in to follow this  

Please Help, Weird Pixel Shader Results...

This topic is 4038 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hey there, I'm doing a conditional operation that if true makes the output alpha of my pixel shader 0.5. And if false 1.0. I'm getting alpha where there shouldn't be so I used pix to debug the problem area. There is only one primitive drawn in the area. PIX (and visually inspected) says (in color values (a,r,g,b)): Render Target Clear (1,0,0,0) Pixel Shader Output (0.502,0.424,0.933,0) Final Frame Buffer (0.749,0.212,0.467,0) When I inspect the shader I see that it actually outputs an alpha value of 1 (which is what it should be). So why is it outputting a completely different value? I can't figure this out, is something wrong. Please help, Devin

Share this post


Link to post
Share on other sites
Advertisement
How pixel shader output alpha applies to the framebuffer is hugely dependent on how the device's render state is configured.

That said, if PIX says the pixel shader output has .5 for its alpha component, why do you think it's actually outputting 1? The reason this seems relevant is that the final frame buffer values seem to be about half of the color component outputs from the pixel shader, which would be about right if doing standard alpha blending.

One way of visually debugging this sort of thing is to force the alpha value to 1, and move the computed alpha into rgb - that way your pixels get written with the computed alpha values, letting you quickly see what the results are for all the pixels you're writing.

Share this post


Link to post
Share on other sites
In pix you can see the register values and step thru a pixel shader for any given rendered pixel in a frame. As I stepped thru I verified that the operations were being performed correctly. Each step verifying the register values and operations. It also tells you the output register values. The output alpha was 1.0.

But it also shows you what the final pixel shader output is in a color with rgba values below it, which was 0.5. This makes no sense because they should be the same shouldn't they?

Thanks for your help,

Devin

Share this post


Link to post
Share on other sites
Quote:
Original post by devronious
But it also shows you what the final pixel shader output is in a color with rgba values below it, which was 0.5. This makes no sense because they should be the same shouldn't they?
There is another stage in the pipeline between the PS output and the actual value in the frame buffer - fixed function blending (now known as the "Output Merger" in D3D10). Depending on how your pipeline is configured it can and will alter the values as written to the RT and eventually displayed on screen - this is what JasonBlochowiak is hinting at [smile]

Use PIX to verify your pipeline configuration at the time your Draw call was submitted. You may think you're setting things up correctly, but its all too easy to let an erronous render state slip through the net and get these sorts of problems.

Blending states, write masks, I/O formats etc...

hth
Jack

Share this post


Link to post
Share on other sites
Thanks I wasn't aware, but after you guys brought up this I can see that it was alpha blending. I also found the problem, which was that as I passed the clip data thru the color registers from the vs to the ps it was rounding the float to a byte value and was throwing me off.

When I switched my ps version to 3.0 from 2.0 it was fixed. Is there a way to allow more significance to go thru the color registers of ps2.0?

Thanks,

Devin

Share this post


Link to post
Share on other sites
To my knowledge, the 2 color registers of a ShaderModel 2 pipeline are still low precision (10 or 12 bits) and clamped to a range of (0, 1). You'll have to use one of the texture coord registers instead if you need the full floating point range and precision.

Share this post


Link to post
Share on other sites
Thanks,

If I use the texture coordinate won't it interpolate and change per pixel? I need to keep it to one value for all pixels.

-Devin

Share this post


Link to post
Share on other sites
Both the texture coordinates and colors are interpolators. If you need a constant value, use a constant register.

Share this post


Link to post
Share on other sites
Schrompf,

Ended up that I could use the Texture Coordinates after all.

Thanks guys, your big help!

tok_junior,

I'm having trouble figuring out how to put data into a fixed register. Perhaps I have to do that using asm lang?

-Devin

Share this post


Link to post
Share on other sites
Quote:
Original post by devronious
tok_junior,

I'm having trouble figuring out how to put data into a fixed register. Perhaps I have to do that using asm lang?
Maybe tok_junior will clarify, but my interpretation of their post is that they've misunderstood your problem - using a CB won't help you here.

hth
Jack

Share this post


Link to post
Share on other sites
Quote:
Original post by jollyjeffers
Quote:
Original post by devronious
tok_junior,

I'm having trouble figuring out how to put data into a fixed register. Perhaps I have to do that using asm lang?
Maybe tok_junior will clarify, but my interpretation of their post is that they've misunderstood your problem - using a CB won't help you here.

hth
Jack


I believe that he's trying to pass a constant from the VS to the PS, correct? "I need to keep it to one value for all pixels" implies that.
Anyway, the correct way of doing it would be to fetch the constant-table of your shader, and using that look up the index of your variable. Then use SetPixelShaderConstantX() to set the value.
I would post some code, but it's late and I'm sure someone else can do that (jollyjeffers perhaps? ;) ). If not, i'll post some in the morning.

Share this post


Link to post
Share on other sites
The vertex data that contains the register information is in one long vertex buffer that draws all at once. I don't beleive I would benefit from externally setting register values. The set must occure within the vertex shader for each vertex and then passed thru to the pixel shader. I think what jollyjeffers is saying is that you can't pass a register value from the vertex shader to the pixel shader without using in and out registers which are limited in use in the case of pixel shader input. (and output for that matter.)

If you have a work around I would love to be brought to a knowledge of it. That would be super!

-Devin

Share this post


Link to post
Share on other sites
I suggest you design your algorithm in a different way.
Would you mind telling me what you're trying to achieve?

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement