Jump to content
  • Advertisement
Sign in to follow this  
UnicornMafia

HLSL readback weirdness

This topic is 2761 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

hello,
I'm working on an HLSL shader in an fx file.
I have my data type set to D3DFMT_A8R8G8B8
my shader looks like this:

float4 PS(float2 tex0 : TEXCOORD0, float2 tex1 : TEXCOORD1): COLOR
{
float4 ret = float4(0.5,0.5,0.5,0.5);
return ret;
}

On 3 of my machines, when I read this texture back, I get 128,128,128,128
however on one of my machines, I get 64,64,64,64. I have played around with
the numbers, and have determined that it's because the alpha is being multiplied by all the other
values, giving me 0.25,0.25,0.25,0.25 as return values.

is there some kind of render state or something that I can set to make it not do this?

it's very strange to me because the card on which this happens is an nVidia 8700M GT,
but it works properly (ie does not multiply the alpha) on machines with 8800GTX and 7950M GT
(as well as on my ATI radion 5870)

does anyone have an idea why this might be happening?
any help would be appreciated.
thanks!
-thomas

Share this post


Link to post
Share on other sites
Advertisement
Yeah can just disable alpha blending altogether. Are you using native D3D9/C++? XNA? SlimDX?

Share this post


Link to post
Share on other sites
I did
device->SetRenderState(D3DRS_ALPHABLENDENABLE, FALSE);
device->SetRenderState(D3DRS_ALPHATESTENABLE, FALSE);
and it gave me the same results.

also
I was wrong about the values I'm getting...
strangely, the results are as follows:

if I return 0.5,0.5,0.5,0.5
when I read back I get 63,63,63,63
if I return 0.5,0.5,0.5,1.0
when I read back I get 127,127,127,255

why would I get 127 instead of 128?

Share this post


Link to post
Share on other sites
Quote:
Original post by UnicornMafia
...

why would I get 127 instead of 128?


Donno about everything else, but...

0b prefix (0bBlahBlahBlah...) = binary number

1. 255 = 0b11111111
2. Right shift by 1 (a.k.a. divide by 2) = 0b01111111 (Since this is an unsigned number we don't have to preserve the leftmost bit a.k.a. the sign bit)
3. 0b01111111 = 127

128 values = 0 to 127, inclusive
256 values = 0 to 255, inclusive (unsigned int a.k.a. 8 bit color component)

0b11111111 a.k.a. 255 is the biggest number that can be represented using 8 bits. To represent 256 we need 9 bits.

By the way if you ever need to do base conversions Google is awesome. Type in "0b11111111 to decimal" or "127 to binary" into Google for example.

[Edited by - jameszhao00 on December 27, 2010 12:16:46 AM]

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!