I know this thing used to be popular when deferred shading hit the scene, as people would do things like writing linear depth values into RGBA8 render-targets (i.e. splitting a float into 4 8-bit integers, then reconstructing the float afterwards).
I'm trying to do this on SM3, which means no bitwise logic (i.e. |, >>, &, etc).
In my specific case, I'm trying to split a float into four 2-bit values, though the logic should be the same as if I were trying to split it into four 8-bit values.
In both cases below, I'm assuming that the input value data is from 0.0f to 1.0f inclusive.
The code that I keep finding everywhere (converted from 8-bit to 2-bit) is as follows:
//splitting: float4 shift = float4( 1, 4, 16, 64 ); float4 abcd = frac( data * shift ); abcd -= abcd.xxyz * float4(0, (float3)1/4.0); //reconstructing: float reconstructed = dot( abcd, float4(3*64,3*16,3*4,3)/255.0 );However, this code is buggy. It's almost right, and it looks mostly right when using the 8-bit version, but the flaw becomes much more exaggerated when you bring it down to a 2-bit version. The problem is that frac never returns 1.0, so you skip over certain values (causing banding), and you can never encode the input value of 1.0f.
The code that I came up with is as follows, and it was mostly through a process of trial-and-error that I arrived at this result.
//splitting: float4 abcd = frac(data * (255.0/float4(256,64,16,4))); abcd = floor(abcd*4)/3; //reconstructing: float reconstructed = dot( abcd, float4(3*64,3*16,3*4,3)/255.0 );If anyone has any links to other methods, or general advice in this area, please share!
N.B. The magic numbers in use are:
255 = max 8-bit value (2-bit * 4 components)
3 = max 2-bit value
4 = num 2-bit values
1,4,16,64,256 = shift by 0, 2, 4, 6, 8 places.