Jump to content
  • Advertisement
Sign in to follow this  

Masking out bit from float as boolean flag

This topic is 712 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hey guys I'm currently having problems with the following thing:


I have a float variable that I use to store 8bit data [0-255] in a color channel of an R8G8B8A8 render target.

What I want to do is take the MSB of that float variable and use it as a boolean flag. So basically reduce my value range to [0-127].

However something doesn't seem to work quite right. I'm thinking it has to do with floating point being stored / converted differently ?


Here's my hlsl code right now:


output.Target3.b = TranslucencyPower * 0.01f;
uint id = 0;
#ifdef IsSkinShading
  id = 1;

  output.Target3.b = asuint(output.Target3.b) | (id << 7); // Store materialID data in MSB


bool isSkinShading = (asuint(Target3.b) >> 7) & 0x0FFFFFFF;
float TranslucencyPower = (asuint(Target3.b) & 0x0FFFFFFF) * 100.0f;

The boolean flag works perfectly like this. But not the actual variable (the one that stores 'TranslucencyPower').

Any idea how I can retrieve this value correctly ?



?I figured it out !

First of all you need to preserve the float value by using a regular uint-cast instead of asuint().

Then the next important thing is you have to convert to and from float again when storing in the 8bit unorm render target.


Here's the working code:

output.Target3.b = float((uint(TranslucencyPower) | (id << 7)) / 255.0f); // Store materialID data in MSB
float TranslucencyPower = float(uint(Target3.b * 255.0f) & 0x0FFFFFFF);
Edited by lipsryme

Share this post

Link to post
Share on other sites

If you are expecting the bit pattern for a floating point number n to be same as that on an integer value, then you will be pleasantly surprised, they are not the same.

Share this post

Link to post
Share on other sites

Before GPUs were any good at doing integer logic, the pure float version of packing a 7bit fraction and 1bit bool into a 8bit UNORM texture would be something like this (untested):

float encode( float value, float flag )
  // if flag, remap value: 0->128, 1->255
  // else,    remap value: 0->127, 1->0
  return flag > 0 ? (value * (127/255.0) + (128/255.0)) : (-value * (128/255.0) + (128/255.0));
void decode( float enc, out float value, out float flag )
  flag = (enc > 0.5) ? 1.0 : 0.0;
  value = abs(enc*2-1);//will be off by 0.5/255 -- if that's an issue, then do the true reverse of the encoding steps based on whether flag is true or not

These days it's usually only one cycle to do int->float / float-> int casting, and bitwise ops are usually full-rate too, so your version might be fine.

If you are expecting the bit pattern for a floating point number n to be same as that on an integer value, then you will be pleasantly surprised, they are not the same.

I think the bit he forgot to state explicitly is that he's working with 8-bit fixed point data in HLSL, which automatically translates the data into 32-bit float while you're working with it, and automatically converts to 8-bit fixed point when you store results to memory.

Edited by Hodgman

Share this post

Link to post
Share on other sites

ok somehow this is not working still...

Let's see...

I start with a 32bit float variable that I have scaled to be in between [0.0-1.0]

Now if I wanted to set the MSB I would shift this by 31 to the left.

However to do the shift I have to convert this float variable to uint in hlsl.

So I'm guessing doing something like:

float fVariable = 255.0f;
float x = float(uint(fVariable));

...would not guarantee me the same 255.0f in the end. the 255.0f would convert to some 32bit unsinged integer value e.g. 1337 and that value would then become 1337.0f afterwards. So the question is how can I shift my float value by 1 and storing that into the 8bit unorm channel without throwing away the actual float value 0-255 or 0-127 in that case. This gets even trickier when trying to retrieve this value.



Let's try a simple case:

output.Target3.b = 0.0f;
output.Target3.b = float(uint(output.Target3.b * 255.0f) | 0x000000FE) / 255.0f;

Converting the actual float value to uint only works if I multiply by 255 because it will just use this value and cut off the decimals. So the [0-1] value needs to be [0-255].

With that I have my 8bit value (actually 32bit) that I can OR with a mask to set the 8th bit...then divide by 255 to get the normalized float value again [0-1] for storage.

So far so good.

Let's assume that the gpu stores a 32bit value that is < 255 without re-arranging anything we would have values of [0-1] after retrieving it with the MSB marked as 1.

We should be able to retrieve the flag like so:

bool flag = uint(Target3.b * 255.0f) & 0x000000FE;

And this seems to work so far, however retrieving that value does not  :(

float test = float(uint(Target3.b * 255.0f) & 0x00000080) / 255.0f; // doesnt work

I multiply this [0-1] value by 255 and convert to uint to get a value in between [0-255], then OR this by 0x80 (8bit) to ignore the MSB of my (expected) 8bit value.

And in the end convert this to float again

The divide is for outputting it on the screen for debug purposes.

I can't think of a reason why this wouldn't work except that the GPU doesnt store values [0.0-255.0] as 1:1 mapping [0.0-1.0]



??I figured it out ! After going through all of the values in my head / piece of paper and debugging the values via RenderDoc....

This is the final working code:


output.Target3.b = TranslucencyPower;
output.Target3.a = TranslucencyAmbient;
#ifdef IsSkinShading
	output.Target3.b = float(uint(output.Target3.b) | 0x00000080) / 255.0f; // Set MSB to 1
	output.Target3.a = float(uint(output.Target3.a) & 0x0000007F) / 255.0f; // Set MSB to 0


bool materialID_Flag1 = uint(Target3.b * 255.0f) & 0x00000080;
bool materialID_Flag2 = uint(Target3.a * 255.0f) & 0x00000080;
float TranslucencyPower = (uint(Target3.b * 255.0f) & 0x0000007F) / 255.0f;
float TranslucencyAmbient = (uint(Target3.a * 255.0f) & 0x0000007F) / 255.0f;	
Edited by lipsryme

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!