Hi guys,

I was studying through tonemapping operators and exposure control.

I've read several threads on gamedev about those topics and I've also read the MJP article on tonemapping on his blog and John Hable on filmic S curve.

Just one thing Is not clear to me: I understand that exposure control and tonemapping are two different things, but I wonder why someone is still multipling the RGB color by the adapted/tonemapped luminance L straight away.
Shouldn't be more correct to go in CIE color space Yxy, adapt Y to get Y' and then given Y'xy go back to RGB to get the new tonemapped color ?
Also, given that automatic exposure is a different thing with respect to tonemapping (which brings the Luminance values in [0,1] range in Reinhard) why in the Reinhard paper he basically calculates the geometric mean to get the average luminance in the context of a tonemapping process (I mean it's just part of the process but is still not the tonemapping step), that means that the relative luminance then can be used in the context of any tonemapping curve (not just reinhard) to have automatic exposure control ? And therefore shouldn't the relative luminance Lr calulation (the one before Lr / (1 + Lr), which is the actual tonemapped Lt ) consider the interpolated Lavg across frames in its calculation ?

Lr = (L(x,y) / Lavg)*a, where Lavg is the interpolated average scene luminance across frames and a is the key.

So it seems to me that the process of calculating the average scene luminance Lavg and from there the relative luminance Lr can be shared accross the different tonemapping curves and can happen just right before whatever curve is applied (not just Reinhard). Otherwise I can't see a general way to calculate automatic autoexposure.


See If understand then:

1) Get scene in HDR
2) Go from RGB to CIE Yxy
3) calculate average scene Yavg using Reinhard method (sum of logL div N)
4) calculate relative luminance Yr = (Y(x,y) / Yavg)*a (a is the key)
5) calculate autoexposure considering the average scene luminance of the previous frame Lavg(i-1) and the one of the current frame Lavg(i) and interpolate between them using an exponential falloff for example...
6) use the new adapted luminance in 4) to calculate the relative adapted luminance of the pixel (x,y) ?

so basically Lavg is always the interpolated average luminance across frames ?

7) use the Yr in the tonemapping curve to get Yt (i.e. tonemapped luminance):

if Reinhard:

Yt = Yr / (1+Yr)

if filmic curve (uncharted 2 variant):

Yt = ((Yr*(A*Yr+C*B)+D*E)/(Yr*(A*Yr+B)+D*F))-E/F

and more generally if whatever tonemapping function f(x):

f(Yr) = Yt

8) Transform from Yt xy CIE to RGB to get the tonemapped RGB color given its tonemapped luminance Lt which is calculated from the Yr, wihch is in turn calculated from the interpolated average scene luminance Lavg across frames (automatic exposure control).

Pfu, I made it.

Now is this the correct way of calculating autoexposure before applying any tonemapping curve to Yr at all ?

And why someone still apply tonemapping on RGB values straight away knowing that is wrong ? (John Hable said that is not correct to apply the reinhard curve to each rgb channel straight away but at the same time its examples are all doing that, maybe is half correct ?! ) Maybe CIE is more correct but because we can't alpha blend in that space we can live with a less correct solution applying tonemapping on rgb right away. But we have to still interpolate Lavg across frames to have the autoexposure control.

A quite important note: Since I send all my lights in one batch I need to have support for alpha blending, therefore I'm thinking to use CIE Yxy only during post fx and tonemap as being on PC I won't have any problem in fp blending support (as instead happens for other platforms ;) ).
I guess the variant Luv is just convenient if we don't have fp blend support (which on PC is not case anymore by long time). So the idea of accumulating lights in Luv is justified only if the underlying platform doesn't have support for fp render target blending and therefore we are constrained in relying on 8888 unsigned light accumulation buffer right ?

Thanks in advance for any reply

The dot product you are talking about is actually doing the RGB to Yxy conversion, but because all you care about is the luminance it simplifies down to a dot product. However, if you want to properly convert the whole color, RGB, to Yxy and then scale it and then convert it back it is a much more intensive operation. So, both are correct. One contains all the information necessary for RGB -> Yxy -> RGB and one only contains enough for RGB -> Y.

I found the equations here: http://stackoverflow.com/questions/7104034/yxy-to-rgb-conversion If you do the algebra on the conversion functions you'll find what I said to be true.

I forgot to make this clear, but the Y in Yxy represents luminance.

It's actually necessary to apply your tone mapping curve to luminance. This is what Reinhard an a lot of other academic papers propose, since it adjusts "intensity" while still maintaining the relative levels of R, G, and B. When simulating film curves its more common to work with RGB (or whatever your spectrum is) directly, and that's what Hable did with his curve. When you're using Reinhard's method for auto-exposure, you just end up with a scalar ratio after you divide the key value by the average luminance which you can then just multiply with your RGB values directly.

So how you model the autoexposure in the case of a filmic curve if you work solely on RGB and you don't have luminance ?

But it's not correct to multiply RGB with the tonemapped Luminance ... There was a thread on gamedev in which they were converting in Yxy, adjust Y to get Y' and then given Y'xy convert back to RGB, which is different from multiplying RGB straight.

Here is the thread:
http://www.gamedev.n...r-tone-mapping/

Thanks

I forgot to make this clear, but the Y in Yxy represents luminance.

Yeah I know the meaning of everything I just wanted to understand in the context of different tone mapping curve how to model autoexposure and I was trying to understand if the calculation of the relative luminance as it happens in Reinhard was usable also while varying the tone mapping curve (I mean not just using the Reinhard one but also a filmic for example). Since filmic is working on RGB and not on Luminance.

[quote name='MegaPixel' timestamp='1348559831' post='4983501']
Since filmic is working on RGB and not on Luminance.

Still filmic use an exposure level, or better said, make an assumption about the used exposure level of the rendered image. So why not adjusting this exposure level depending on the average luminance level ?
[/quote]

So that means that I can post multiply the RGB color with the adjusted luminance before to run it through the filmic curve or after ?

Also, shouldn't I have to go from CIE to RGB and viceversa and not just post multiply RGB by the adjusted luminance ?

Just to clarify further I'll post the shader code in which I apply Reinhard (you can see uncharted filmic commented);


float4 PSDeferred(PS_INPUT Input) : SV_Target0{
float4 sceneColor = hdrSceneBuffer.Load(int3(Input.screenPos.xy,0)); //get linear hdr color
float4 bloomColor = hdrBloomBuffer.Sample(bloomSampler,Input.tc); //get linear hdr bloom color

//
const float3x3 RGB2XYZ = {0.4124f, 0.3576f, 0.1805f,
0.2126f, 0.7152f, 0.0722f,
0.0193f, 0.1192f, 0.9505f };
//go from RGB color space to CIE
float3 XYZ = mul(RGB2XYZ,sceneColor.rgb);
float den = 1.f/dot(float3(1,1,1),XYZ);
float3 Yxy = float3(XYZ.y,XYZ.xy*den);

//Reinhard tonemapping operator
float averageSceneLuminance = exp(luminanceBuffer.Load(int3(0,0,9)).r);
float fragmentLuminance = dot(sceneColor.rgb,XYZ.y);
float relativeLuminance = (fragmentLuminance / (averageSceneLuminance+0.0001f))*key;

float tonemappedLuminance = (relativeLuminance*(1.f+(relativeLuminance/(Lw*Lw)))) / (1.f+relativeLuminance);
//now go from CIE back to RGB with new tonemapped luminance
const float3x3 XYZ2RGB = {3.2405f, -1.5371f, -0.4985f,
-0.9693f, 1.8760f, 0.0416f,
0.0556f, -0.2040f, 1.0572f };
float s = (tonemappedLuminance/Yxy.z);
XYZ.x = Yxy.y*s;
XYZ.y = tonemappedLuminance;
XYZ.z = (1.f-Yxy.y-Yxy.z)*s;
float4 finalColor = float4(mul(XYZ2RGB,XYZ),1);//sceneColor*tonemappedLuminance;
//

//filmic tonemapping (Uncharted2)
/*finalColor = max(0.f,finalColor-0.004f);
float4 gammaColor = (finalColor*(6.2f*finalColor+0.5f))/(finalColor*(6.2f*finalColor+1.7f)+0.06f);
//out the color applying gamma correction
return gammaColor+pow(bloomColor,1.f/2.2f);*/



return pow(finalColor+bloomColor*2.f,1.f/2.2f);


How I can control autoexposure for the filmic one ?