Jump to content
  • Advertisement
Sign in to follow this  

Packing/unpacking floats<->several channels

This topic is 2833 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Does anyone have any links/tips on packing/unpacking single floats to/from multi-channel texture formats?
I know this thing used to be popular when deferred shading hit the scene, as people would do things like writing linear depth values into RGBA8 render-targets (i.e. splitting a float into 4 8-bit integers, then reconstructing the float afterwards).

I'm trying to do this on SM3, which means no bitwise logic (i.e. |, >>, &, etc).
In my specific case, I'm trying to split a float into four 2-bit values, though the logic should be the same as if I were trying to split it into four 8-bit values.
In both cases below, I'm assuming that the input value [font=courier new,courier,monospace]data[/font] is from [font=courier new,courier,monospace]0.0f[/font] to [font=courier new,courier,monospace]1.0f[/font] inclusive.

The code that I keep finding everywhere (converted from 8-bit to 2-bit) is as follows://splitting:
float4 shift = float4( 1, 4, 16, 64 );
float4 abcd = frac( data * shift );
abcd -= abcd.xxyz * float4(0, (float3)1/4.0);
float reconstructed = dot( abcd, float4(3*64,3*16,3*4,3)/255.0 );
However, this code is buggy. It's almost right, and it looks mostly right when using the 8-bit version, but the flaw becomes much more exaggerated when you bring it down to a 2-bit version. The problem is that frac never returns 1.0, so you skip over certain values (causing banding), and you can never encode the input value of 1.0f.

The code that I came up with is as follows, and it was mostly through a process of trial-and-error that I arrived at this result.//splitting:
float4 abcd = frac(data * (255.0/float4(256,64,16,4)));
abcd = floor(abcd*4)/3;
float reconstructed = dot( abcd, float4(3*64,3*16,3*4,3)/255.0 );

If anyone has any links to other methods, or general advice in this area, please share! smile.png

N.B. The magic numbers in use are:
255 = max 8-bit value (2-bit * 4 components)
3 = max 2-bit value
4 = num 2-bit values
1,4,16,64,256 = shift by 0, 2, 4, 6, 8 places.

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!