Multiply RGB by luminance or working in CIE ? Which is more correct ?

MegaPixel · 2012-09-25T14:20:33

Hi guys, I was studying through tonemapping operators and exposure control. I've read several threads on gamedev about those topics and I've also read the MJP article on tonemapping on his blog and John Hable on filmic S curve. Just one thing Is not clear to me: I understand that exposure control and tonemapping are two different things, but I wonder why someone is still multipling the RGB color by the adapted/tonemapped luminance L straight away. Shouldn't be more correct to go in CIE color space Yxy, adapt Y to get Y' and then given Y'xy go back to RGB to get the new tonemapped color ? Also, given that automatic exposure is a different thing with respect to tonemapping (which brings the Luminance values in [0,1] range in Reinhard) why in the Reinhard paper he basically calculates the geometric mean to get the average luminance in the context of a tonemapping process (I mean it's just part of the process but is still not the tonemapping step), that means that the relative luminance then can be used in the context of any tonemapping curve (not just reinhard) to have automatic exposure control ? And therefore shouldn't the relative luminance Lr calulation (the one before Lr / (1 + Lr), which is the actual tonemapped Lt ) consider the interpolated Lavg across frames in its calculation ? Lr = (L(x,y) / Lavg)*a, where Lavg is the interpolated average scene luminance across frames and a is the key. So it seems to me that the process of calculating the average scene luminance Lavg and from there the relative luminance Lr can be shared accross the different tonemapping curves and can happen just right before whatever curve is applied (not just Reinhard). Otherwise I can't see a general way to calculate automatic autoexposure. See If understand then: 1) Get scene in HDR 2) Go from RGB to CIE Yxy 3) calculate average scene Yavg using Reinhard method (sum of logL div N) 4) calculate relative luminance Yr = (Y(x,y) / Yavg)*a (a is the key) 5) calculate autoexposure considering the average scene luminance of the previous frame Lavg(i-1) and the one of the current frame Lavg(i) and interpolate between them using an exponential falloff for example... 6) use the new adapted luminance in 4) to calculate the relative adapted luminance of the pixel (x,y) ? so basically Lavg is always the interpolated average luminance across frames ? 7) use the Yr in the tonemapping curve to get Yt (i.e. tonemapped luminance): if Reinhard: Yt = Yr / (1+Yr) if filmic curve (uncharted 2 variant): Yt = ((Yr*(A*Yr+C*B)+D*E)/(Yr*(A*Yr+B)+D*F))-E/F and more generally if whatever tonemapping function f(x): f(Yr) = Yt 8) Transform from Yt xy CIE to RGB to get the tonemapped RGB color given its tonemapped luminance Lt which is calculated from the Yr, wihch is in turn calculated from the interpolated average scene luminance Lavg across frames (automatic exposure control). Pfu, I made it. Now is this the correct way of calculating autoexposure before applying any tonemapping curve to Yr at all ? And why someone still apply tonemapping on RGB values straight away knowing that is wrong ? (John Hable said that is not correct to apply the reinhard curve to each rgb channel straight away but at the same time its examples are all doing that, maybe is half correct ?! ) Maybe CIE is more correct but because we can't alpha blend in that space we can live with a less correct solution applying tonemapping on rgb right away. But we have to still interpolate Lavg across frames to have the autoexposure control. A quite important note: Since I send all my lights in one batch I need to have support for alpha blending, therefore I'm thinking to use CIE Yxy only during post fx and tonemap as being on PC I won't have any problem in fp blending support (as instead happens for other platforms ;) ). I guess the variant Luv is just convenient if we don't have fp blend support (which on PC is not case anymore by long time). So the idea of accumulating lights in Luv is justified only if the underlying platform doesn't have support for fp render target blending and therefore we are constrained in relying on 8888 unsigned light accumulation buffer right ? Thanks in advance for any reply

MegaPixel

241

Author

September 25, 2012 01:03 PM

[quote name='MegaPixel' timestamp='1348574474' post='4983535']
So that means that I can post multiply the RGB color with the adjusted luminance before to run it through the filmic curve or after ?

It is the question of what you want to archieve.

If you just want to fake the human eye (adjusting to the darkness), then it would be worth an attempt. According to the filmic tonemapper function here, I would just alter the ExposureBias according to the average luminance. It is more like playing around with the mapper until it looks good vs. a physical correct solution.
[/quote]

so that means that I should use



float  relativeLuminance  = (fragmentLuminance / (averageSceneLuminance+0.0001f))*key;

to modulate the color without run it into the Reinhard tonemapper:



float  tonemappedLuminance   = (relativeLuminance*(1.f+(relativeLuminance/(Lw*Lw)))) / (1.f+relativeLuminance);).

so for the filmic curve should be:



finalColor				*= relativeLuminance;

finalColor				 = max(0.f,finalColor-0.004f);

float4 gammaColor  = (finalColor*(6.2f*finalColor+0.5f))/(finalColor*(6.2f*finalColor+1.7f)+0.06f);

//out the color applying gamma correction

return gammaColor+pow(bloomColor,1.f/2.2f);

(no CIE to RGB needed in this case as the filmic curves works in RGB space straight away)

MegaPixel

241

Author

September 25, 2012 01:12 PM

[quote name='MegaPixel' timestamp='1348574474' post='4983535']
So that means that I can post multiply the RGB color with the adjusted luminance before to run it through the filmic curve or after ?

It is the question of what you want to archieve.

If you just want to fake the human eye (adjusting to the darkness), then it would be worth an attempt. According to the filmic tonemapper function here, I would just alter the ExposureBias according to the average luminance. It is more like playing around with the mapper until it looks good vs. a physical correct solution.

Simple example (don't know if it look good):



ExposureBias = 1.5 + (1.0-avgLumi);

[/quote]

I understand the idea of tweaking etc. but I'd like first to understand the correct underlying theory and then tweak from there. If you work on the avgLum directly means that you are not using the scaled one L(x,y) / avgLum which is the pixel relative luminance (assume key =1 ) ... see Reinhard for reference

CryZe

773

September 25, 2012 01:19 PM

I would just alter the ExposureBias according to the average luminance.

Don't do that. The exposure bias is part of Hable's tone mapping operator. It's there to assure that the average luminance (1) gets mapped to 0.5. Otherwise it would be way brighter. You don't need that with other tone mapping operators though.

BTT: I'm not quite sure what you guys are doing. It's this easy:



float3 color = sampleColor(texCoord);

float3 bloomColor = sampleBloom(texCoord);



//Add the bloom (bloomFactor is some constant, 0.0075f is a good value imho)

color += bloomFactor * bloomColor;



//Calculate the exposure (this is the simplest way to do it)

float exposure = 1 / averageLuminance;



//Adjust the exposure

float3 color *= exposure;



//Map the color to LDR (Serr tone mapping operator, the same as Hables standard values)

float3 tonemappedColor = 4.6f * color / (3.6f * color + 5.6f);



//Adjust gamma and return (ajusting gamma might not be needed)

return pow(tonemappedColor, 1/2.2f);

MegaPixel

241

Author

September 25, 2012 01:23 PM

[quote name='Ashaman73' timestamp='1348577752' post='4983548']
I would just alter the ExposureBias according to the average luminance.

Don't do that. The exposure bias is part of Hable's tone mapping operator. It's there to assure that the average luminance (1) gets mapped to 0.5. Otherwise it would be way brighter. You don't need that with other tone mapping operators though.

BTT: I'm not quite sure what you guys are doing. It's this easy:



float3 color = sampleColor(texCoord);

float3 bloomColor = sampleBloom(texCoord);



//Add the bloom (bloomFactor is some constant, 0.0075f is a good value imho)

color += bloomFactor * bloomColor;



//Calculate the exposure (this is the simplest way to do it)

float exposure = 1 / averageLuminance;



//Adjust the exposure

float3 color *= exposure;



//Map the color to LDR (Serr tone mapping operator, the same as Hables standard values)

float3 tonemappedColor = 4.6f * color / (3.6f * color + 5.6f);



//Adjust gamma and return (ajusting gamma might not be needed)

return pow(tonemappedColor, 1/2.2f);

[/quote]

you run the tonemapping process even whem you have to perform the bright pass ? I guess yes since you want to cut the luminance under a given threshold value... ?

MegaPixel

241

Author

September 25, 2012 01:25 PM

[quote name='CryZe' timestamp='1348579184' post='4983559']
[quote name='Ashaman73' timestamp='1348577752' post='4983548']
I would just alter the ExposureBias according to the average luminance.

Don't do that. The exposure bias is part of Hable's tone mapping operator. It's there to assure that the average luminance (1) gets mapped to 0.5. Otherwise it would be way brighter. You don't need that with other tone mapping operators though.

BTT: I'm not quite sure what you guys are doing. It's this easy:



float3 color = sampleColor(texCoord);

float3 bloomColor = sampleBloom(texCoord);



//Add the bloom (bloomFactor is some constant, 0.0075f is a good value imho)

color += bloomFactor * bloomColor;



//Calculate the exposure (this is the simplest way to do it)

float exposure = 1 / averageLuminance;



//Adjust the exposure

float3 color *= exposure;



//Map the color to LDR (Serr tone mapping operator, the same as Hables standard values)

float3 tonemappedColor = 4.6f * color / (3.6f * color + 5.6f);



//Adjust gamma and return (ajusting gamma might not be needed)

return pow(tonemappedColor, 1/2.2f);

[/quote]
[/quote]

you run the tonemapping process even whem you have to perform the bright pass ? I guess yes since you want to cut the luminance under a given threshold value... ?
Also can you point me to some good reading to understand first the simplest way instead of just applying Reinhard or whatever other tonemapper blindly ? thanks in advance for your help ;)
Btw I tried your serr approach and my scene looked way too much bright and I have just one pointlight !

CryZe

773

September 25, 2012 01:31 PM

you run the tonemapping process even whem you have to perform the bright pass ? I guess yes since you want to cut the luminance under a given threshold value... ?

There's no need for a bright pass with a threshold value if you actually have high dynamic range values. Threshold values for bloom are from back in the days where you couldn't afford the bandwidth for HDR illumination. Bright lights are thousands if not millions of times brighter than the rest of your scene (or at least they should

). By simply using a multiplicative factor you reduce the amount of bloom so that only the bloom of these really bright lights is visible. So there's no need for either a bright pass or a threshold value for bloom. A threshold value is just not realistic. Lenses always scatter a tiny percentage of the light onto nearby pixels (depending on the quality of the lenses), they never subtract light.

MegaPixel

241

Author

September 25, 2012 01:35 PM

[quote name='MegaPixel' timestamp='1348579435' post='4983562']
you run the tonemapping process even whem you have to perform the bright pass ? I guess yes since you want to cut the luminance under a given threshold value... ?

There's no need for a bright pass with a threshold value if you actually have high dynamic range values. Threshold values for bloom are from back in the days where you couldn't afford the bandwidth for HDR illumination. Bright lights are thousands if not millions of times brighter than the rest of your scene (or at least they should

). By simply using a multiplicative factor you reduce the amount of bloom so that only the bloom of these really bright lights is visible. So there's no need for either a bright pass or a threshold value for bloom.
[/quote]

So you scale down the color intensity by some factor before gaussian blur is applied (like during the downsampling of the scene something like sceneColor*threshold)? But it doesn't seem to be effective, because what I actually want is for only some part of the scene to glow and I don't want to see all the scene blurred ...

CryZe

773

September 25, 2012 01:38 PM

It doesn't matter whether you do it before or afterwards. I do it when combining it with the source color again. But the result is the same either way. Not everything will be blurred. Like I said, with proper high dynamic range, it won't happen. Simply decrease the factor to something where not everything is glowing and increase the luminance of everything you want to see glowing (you might need to make them thousands of times brighter, but this is actually realistic). And that's why you need the tone mapper. The tone mapper will reduce their perceived luminance and lets them look like they wouldn't actually be thousands of times brighter.

The sun in my engine is for example 6.5 million lux while shadows on a sunny day are just 10,000 lux, nights are 0.005 lux.

MegaPixel

241

Author

September 25, 2012 02:20 PM

It doesn't matter whether you do it before or afterwards. I do it when combining it with the source color again. But the result is the same either way. Not everything will be blurred. Like I said, with proper high dynamic range, it won't happen. Simply decrease the factor to something where not everything is glowing and increase the luminance of everything you want to see glowing (you might need to make them thousands of times brighter, but this is actually realistic). And that's why you need the tone mapper. The tone mapper will reduce their perceived luminance and lets them look like they wouldn't actually be thousands of times brighter.

The sun in my engine is for example 6.5 million lux while shadows on a sunny day are just 10,000 lux, nights are 0.005 lux.

How you define a lux in your engine ?

Also to have good bloom I was thinking to downsample 5 times, something like:

1) blur while downsampling the scene for the first time
2) blur while downsampling to the next level
...
till 5) ...

then:

upsample each downscaled version of the previous 5 steps JUST to the previous level (so don't stretch a given level to fullscreen?) looks like Epic is doing like this in unreal engine 4 ...
I just have the doubt that they actually stretch each level to fullscreen while adding them together ... can you confirm on that ?

Multiply RGB by luminance or working in CIE ? Which is more correct ?

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Multiply RGB by luminance or working in CIE ? Which is more correct ?

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines