Sign in to follow this  

HLSL readback weirdness

This topic is 2546 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

hello,
I'm working on an HLSL shader in an fx file.
I have my data type set to D3DFMT_A8R8G8B8
my shader looks like this:

float4 PS(float2 tex0 : TEXCOORD0, float2 tex1 : TEXCOORD1): COLOR
{
float4 ret = float4(0.5,0.5,0.5,0.5);
return ret;
}

On 3 of my machines, when I read this texture back, I get 128,128,128,128
however on one of my machines, I get 64,64,64,64. I have played around with
the numbers, and have determined that it's because the alpha is being multiplied by all the other
values, giving me 0.25,0.25,0.25,0.25 as return values.

is there some kind of render state or something that I can set to make it not do this?

it's very strange to me because the card on which this happens is an nVidia 8700M GT,
but it works properly (ie does not multiply the alpha) on machines with 8800GTX and 7950M GT
(as well as on my ATI radion 5870)

does anyone have an idea why this might be happening?
any help would be appreciated.
thanks!
-thomas

Share this post


Link to post
Share on other sites
I did
device->SetRenderState(D3DRS_ALPHABLENDENABLE, FALSE);
device->SetRenderState(D3DRS_ALPHATESTENABLE, FALSE);
and it gave me the same results.

also
I was wrong about the values I'm getting...
strangely, the results are as follows:

if I return 0.5,0.5,0.5,0.5
when I read back I get 63,63,63,63
if I return 0.5,0.5,0.5,1.0
when I read back I get 127,127,127,255

why would I get 127 instead of 128?

Share this post


Link to post
Share on other sites
Quote:
Original post by UnicornMafia
...

why would I get 127 instead of 128?


Donno about everything else, but...

0b prefix (0bBlahBlahBlah...) = binary number

1. 255 = 0b11111111
2. Right shift by 1 (a.k.a. divide by 2) = 0b01111111 (Since this is an unsigned number we don't have to preserve the leftmost bit a.k.a. the sign bit)
3. 0b01111111 = 127

128 values = 0 to 127, inclusive
256 values = 0 to 255, inclusive (unsigned int a.k.a. 8 bit color component)

0b11111111 a.k.a. 255 is the biggest number that can be represented using 8 bits. To represent 256 we need 9 bits.

By the way if you ever need to do base conversions Google is awesome. Type in "0b11111111 to decimal" or "127 to binary" into Google for example.

[Edited by - jameszhao00 on December 27, 2010 12:16:46 AM]

Share this post


Link to post
Share on other sites

This topic is 2546 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this