🔒 Locked

LogLuv Encoding For HDR

Started by MJP Jul 5, 2008 at 1:35 PM 14 replies 16.2k views

Original Post

Jul 5, 2008 at 1:35 PM

For my recent project (targeting the 360 and PC through XNA) I've been doing some experimenting with alternative formats for HDR. If I were just doing PC I'd probably just use fp16 and be done with it, but for the 360 I've been trying out some alternatives since fp16 has some disadvantages on that platfrom (and since the special fp10 frame buffer format isn't available through XNA [flaming]) Anyway my most recent adventure has been with LogLuv encoding (Heavenly Sword's famous NAO32 technique). I've managed to come up with a working implmentation based on Christer Ericson's optimized Cg code, however I've been getting artifacts in the final result:

I'm guessing that I'm running into discontinuites somewhere...however I can't seem to figure out where. If anyone has any experience with this and could point in the right direction, I'd be eternally grateful. This is the shader code I'm using:


// M matrix, for encoding
const static float3x3 M = float3x3(
    0.2209, 0.3390, 0.4184,
    0.1138, 0.6780, 0.7319,
    0.0102, 0.1130, 0.2969);

// Inverse M matrix, for decoding
const static float3x3 InverseM = float3x3(
	6.0013,	-2.700,	-1.7995,
	-1.332,	3.1029,	-5.7720,
	.3007,	-1.088,	5.6268);	

float4 LogLuvEncode(in float3 vRGB) 
{		 
    float4 vResult; 
    float3 Xp_Y_XYZp = mul(vRGB, M);
    Xp_Y_XYZp = max(Xp_Y_XYZp, float3(1e-6, 1e-6, 1e-6));
    vResult.xy = Xp_Y_XYZp.xy / Xp_Y_XYZp.z;
    float Le = 2 * log2(Xp_Y_XYZp.y) + 128;
    vResult.z = Le / 256;
    vResult.w = frac(Le);
    return vResult;
}

float3 LogLuvDecode(in float4 vLogLuv)
{	
	float Le = vLogLuv.z * 256 + vLogLuv.w;
	float3 Xp_Y_XYZp;
	Xp_Y_XYZp.y = exp2((Le - 128) / 2);
	Xp_Y_XYZp.z = Xp_Y_XYZp.y / vLogLuv.y;
	Xp_Y_XYZp.x = vLogLuv.x * Xp_Y_XYZp.z;
	float3 vRGB = mul(Xp_Y_XYZp, InverseM);
	return max(vRGB, 0);
}

I've tried playing around with clamping the values of Xp_Y_XYZp to 1e-6 and also the value of vLogLuv.y before division, but it hasn't gotten rid of the artifacts. I know Marco (Heavenly Sword dev) mentioned you had be careful to avoid carry problems if you're linearly filtering, however I'm getting these artifacts even when I just do a straight decode of the whole render target without bloom or tone-mapping. So yeah...I'm basically stumped. Any mathematical geniuses out there who can help out a poor soul? [smile] By the way, this is the original paper describing the LogLuv format.

remigius

Jul 7, 2008 at 3:38 AM

(Sorry for the awkward read, this post kinda evolved as I tinkered along :)

I haven't worked with this myself yet, but from those resources you posted I couldn't see anything wrong with your implementation. I fixed up a little app to test your LogLuv encoding/decoding function on a plain image (just a pixel shader doing RGB->LogLuv->RGB) and this was the result:

According to the original LogLuv paper, RGB->LogLuv->RGB should yield an almost identical bitmap (p4, 4.4), so I'd guess there's indeed something wrong with either the encoding/decoding or my copy-paste skills. I'm not sure if my results are even remotely conclusive though (does it make sense to render decoded RGB imagery without tonemapping?).

Anyway, in the comments section of Christers blog NAO posted his approach and I noticed he went about packing the MSBs of Le differently than Christer. Makes sense too, since Christers implementation seems to depend on precission limitations to clamp the MSB. NAO on the other hand explicitly clamps this part of Le. Long story short, changing these lines in your encoding

vResult.z = Le / 256;vResult.w = frac(Le);

to this

vResult.w = frac(Le);vResult.z = (Le - (floor(vResult.w*256.0f))/256.0f)/256.0f;

stops the color bleeding in my test app and might also solve your artifacts.

davepermen

Jul 7, 2008 at 9:31 AM

looks like skyroads? sorry to be offtopic, but i really love skyroads, so the picture (which looks great btw) got me interested

MJP

Jul 7, 2008 at 10:15 AM

Thanks for taking some time to investigate Remigius, you're a prince. [smile] I'll try that out when I get home, see what results I get. Even if it just eliminates some of that banding it'll be some good progress.

Quote:
Original post by davepermen
looks like skyroads? sorry to be offtopic, but i really love skyroads, so the picture (which looks great btw) got me interested

Nice catch there! The game I'm making is actually going to be something of a Skyroads clone.

remigius

Jul 7, 2008 at 4:51 PM

NP, I came across LogLuv on Wolfs blog and didn't quite know what to make of it, so I'm glad I had a chance to tinker with it. For future reference, using NAO's approach completely removes all banding and there is no longer any visible difference between the original RGB image and the RGB -> LogLuv -> RGB one. Maybe Christers trick (Le / 256) only works on PS3 hardware?

Now let's hope it solves you artifacts as well [smile]

MJP

Jul 7, 2008 at 5:40 PM

Hmm...using that method seems to work alright when rendering to a fp16 surface, but I still get the same artifacts and banding with A8R8G8B8. I suppose it's a precision problem then...but I'm not sure why I'm having one. Oh well, more investigation is needed...

Thank you anyway, remigius.

remigius

Jul 8, 2008 at 12:20 AM

Ugh, I was thinking it should work with ARGB surfaces since that's what my app is using, but obviously there aren't any precission issues when doing the RGB -> LogLuv -> RGB test in an single pixelshader. I'll try messing around with the test app later today using an intermediate render target, hopefully I can find out something useful. I'm not sure it's a precission issue though, since I did a quick check with this:

    float4 lle = LogLuvEncode(color.rgb);        lle *= 256;    lle = floor(lle);    lle /= 256;        float3 lld = LogLuvDecode(lle);

And that does work (only slightly less saturated, but no banding or other artifacts), so it would seem the approach is sound. That at least narrows down the problem to either the conversion/rendering to an RGB surface or some weird filtering issue. Have you tried using point filtering to test how that works out?

Anyway... [smile]

I had a quick go at it and there was indeed some banding popping up again when doing RGB -> LogLuv -> (backbuffer resolve) -> LogLuv -> RGB. I had been wondering if the 128 & 256 numbers were valid for the 0-255 color range, so I decided to change those to 127 & 255 and that actually removes the banding even when using the intermediate buffer. Guess it was more of a silly range issue than precission. This code seems to do the trick:

// M matrix, for encodingconst static float3x3 M = float3x3(    0.2209, 0.3390, 0.4184,    0.1138, 0.6780, 0.7319,    0.0102, 0.1130, 0.2969);// Inverse M matrix, for decodingconst static float3x3 InverseM = float3x3(	6.0013,	-2.700,	-1.7995,	-1.332,	3.1029,	-5.7720,	.3007,	-1.088,	5.6268);	float4 LogLuvEncode(in float3 vRGB) {		     float4 vResult;     float3 Xp_Y_XYZp = mul(vRGB, M);    Xp_Y_XYZp = max(Xp_Y_XYZp, float3(1e-6, 1e-6, 1e-6));    vResult.xy = Xp_Y_XYZp.xy / Xp_Y_XYZp.z;    float Le = 2 * log2(Xp_Y_XYZp.y) + 127;    vResult.w = frac(Le);    vResult.z = (Le - (floor(vResult.w*255.0f))/255.0f)/255.0f;    return vResult;}float3 LogLuvDecode(in float4 vLogLuv){		float Le = vLogLuv.z * 255 + vLogLuv.w;	float3 Xp_Y_XYZp;	Xp_Y_XYZp.y = exp2((Le - 127) / 2);	Xp_Y_XYZp.z = Xp_Y_XYZp.y / vLogLuv.y;	Xp_Y_XYZp.x = vLogLuv.x * Xp_Y_XYZp.z;	float3 vRGB = mul(Xp_Y_XYZp, InverseM);	return max(vRGB, 0);}

Ps. for giggles I tried to see if Christers shortcut (Le / 255) worked this way, but that's still producing massive amounts of banding. I'm really curious to know if no one else ran into this or if it's simply no problem on PS3 hardware.

[Edited by - remigius on July 8, 2008 1:20:45 AM]

MJP

Jul 8, 2008 at 11:10 PM

Okay this is really really really embarrassing...

It turns out I had alpha-testing enabled and a few scattered pixels were getting killed when the fractional component came out to zero. After turning that off and using the code remigius came up with it all comes out very nice with no banding. Thanks again for the help, rim.

Marco Salvi

Jul 9, 2008 at 9:18 AM

Good, problem solved! I guess I will need to write a post on my blog about NAO32/LogLuv encoding sooner or later..

wolf

Jul 11, 2008 at 12:45 PM

I actually skipped the log part and stored the luminance value in two 8-bit channels ... this is much much faster and looks quite cool.

MJP

Jul 11, 2008 at 1:01 PM

Quote:
Original post by wolf
I actually skipped the log part and stored the luminance value in two 8-bit channels ... this is much much faster and looks quite cool.

Interesting...does it still look okay with multi-sampling or other linear filtering? I suppose I'll just have to try it myself when I get home...

Marco Salvi

Jul 11, 2008 at 1:02 PM

Quote:
Original post by wolf
I actually skipped the log part and stored the luminance value in two 8-bit channels ... this is much much faster and looks quite cool.

Yep, that works as well if you don't need to support insane dynamic ranges :)
Actually it's also possible to save the alpha channel for something else and encode logarithmic luminance in just 8 bits. If one wants so support a medium dynamic range the final results are totally acceptable.

jstroh

Jul 22, 2008 at 3:10 PM

I wonder MJP if you might explain the whole process of using LogLUV/downsampling/etc to get the final glow on your ship and on the road?

MJP

Jul 22, 2008 at 3:30 PM

Sure, no problem. What I have going now is basically this:

1. Render the scene to an R8G8B8A8 texture, encoded in LogLuv (using the LogLuvEncode function)

2. Downscale to 1/16th size
-grab 16 samples
-decode samples using LogLuvDecode
-combine the samples and divide by 16
-encode the samples

3. Decode, threshold, and re-encode

4. Vertical Gaussian blur, similar to downscale (decode, filter, re-encode)

5. Horizontal Gaussian blur, don't re-encode (output linear RGB)

6. Combine with original texture

Obviously I'm doing a lot of encoding/rencoding here which removes some of the benefit of working with LogLuv, so when I have some time I'm going to experiment with filtering in linear space with one or some of those steps. When I just did everything in linear space the results weren't that great, so it'll probably take some trial and error to figure out.

jstroh

Jul 22, 2008 at 3:40 PM

So those textures that are blurred just have a value higher than the threshold?

MJP

Jul 23, 2008 at 12:35 AM

Quote:
Original post by jstroh
So those textures that are blurred just have a value higher than the threshold?

I just do a very simple threshold, like this:

float4 ThresholdPS (	in float2 in_vTexCoord    : TEXCOORD0,			uniform bool bUseNAO32	) : COLOR0 {    float4 vSample = tex2D(PointSampler0, in_vTexCoord);	    if (bUseNAO32)        vSample = float4(LogLuvDecode(vSample), 1.0f);		    vSample -= g_fThreshold;    vSample = max(vSample, 0.0f);    	    if (bUseNAO32)        vSample = LogLuvEncode(vSample.rgb);	    return vSample;}

Then I feed the result of that pass into the blur shaders. There's obviously a lot fancier things you could do, but for me this is working well enough for now.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.