Sign in to follow this  

Diffuse IBL - Importance Sampling vs Spherical Harmonics

This topic is 677 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hey guys,

 

I'm currently implementing a real-time physically based rendering (PBR) pipeline with image based lighting (IBL) and ran into a rather irritating issue.

 

For IBL specular I use an importance sampling approach with a pre-integrated environment and LUT as introduced by Karis for Unreal. With the addition of sampling multiple mip levels during pre-integration to reduce aliasing (originally from GPU Gems 3. Example code listing here). This works perfectly and I'm loving the results. 

 

Now for diffuse I started out with trying the same approach (pre-integration + LUT) as I saw a few documents choosing this over Spherical Harmonics (SH) (Frostbite for example, at least in their paper). I also have no experience with SH so I preferred to start out with the same workflow as the specular part. The sampling is all set up and functional, but the amount of aliasing is immense. Given almost any (HDR) input, the strong light pixels will just be repeated across the integrated texture. Even with pdf-based filtering as in the specular component, the strong dots cannot be averaged out to continuous spot of light (except after an immense amount of samples).

 

I understand that a stronger amount of aliasing can be natural, as the diffuse sampling is not as focused as the specular one. Certainly for some of my test HDR files with a sun sput of 20 000 compared to other values of around 0-20. But I was wondering if I forgot something crucial to lower the amount of needed samples for a robust diffuse result texture, like maybe prefiltering or only sampling from low mip levels (this btw does solve the problem, but seems very hacky to me). As of now thousands of samples are needed (unless sampling only the very low mip levels of the environment map). The resulting image in that case is correct however, so I am absolutely certain that my other components work as intended.

 

Also, since this has me rather irritated, I started thinking about Spherical Harmonics as replacement for diffuse importance sampling. Do any of you have experience with that? How does it compare visually to the real-time sampling results (if they work as intended)? Is there a preference towards Spherical Harmonics for other reasons than smaller storage requirements?

 

Greetings

Edited by Kevin Van Nerum

Share this post


Link to post
Share on other sites

So I'm correct in assuming that you're using the same approach as your specular where you're sampling off of a fairly high resolution HDR cubemaps for your diffuse results?

If so, don't bother, because as you've figured out for yourself that will require a massive amount of samples to get working nicely as you're sampling the entire hemisphere rather than a focused area as you would with glossy specular materials.

 

What you could do here is filter and downsample a seperate HDR cubemap offline just like how you would approach filtering your specular mip levels for high roughness values, but just for your diffuse term. You can use a similar importance sampling approach to do this. This gives you a fairly tiny texture which should give very acceptable results with only a minimal amount of samples required. At this point you should consider just doing an SH representation though as that will give pretty much the exact same results (as you're dealing with very low frequency data anyway) with a lower memory footprint, but if you're not familiar with that just yet you can experiment with just the separate diffuse texture.

 

Whether you use an SH approach or a texture-based one will not make a huge change in the end result as we're dealing with very low frequency data here anyway. Spherical harmonics are just another way to represent your data, they're not a different way of generating that data. You'll still need to sample your diffuse data somehow before you can store it.

 

Additionally you won't need any type of LUT as your diffuse term is usually solely dependent on your normal vector, unless you're doing some expensive fancy diffuse model which is view-dependent or which takes surface roughness into account. Even if that would be the case, this usually results down to taking your fresnel factor into account somewhere which can be accounted for after sampling your diffuse IBL.

 

(PS: Dag Kevin, lang geleden!)

Share this post


Link to post
Share on other sites

Most often HDR sky textures don't contain sun, as it's a separate analytic light source and you don't want to apply it twice to the scene. Therefore you usually don't have to deal with such extreme values and ~32-64 samples are enough for diffuse pre-integration using importance sampling. It's also pretty cheap, as 6x8x8 destination pixels are enough for low frequency data like diffuse. This of course doesn't yield perfect results, but errors are hard to notice in a real scene with textures.

 

If you are interested in diffuse pre-integration using SH, then check out this great post by Sébastien Lagarde (it also contains an example application with source code):

https://seblagarde.wordpress.com/2012/06/10/amd-cubemapgen-for-physically-based-rendering/

Share this post


Link to post
Share on other sites
Spherical harmonics are pretty popular for representing environment diffuse, because they're fairly compact and they can be evaluated with just a bit of shader code. For L2 SH you need 9 coefficients (so 27 floats for RGB), which is about the size of a 2x2 cubemap. However it will always introduce some amount of approximation error compared to ground truth, and in some cases that error can be very noticeable. In particular it has the problem where very intense lighting will cause the SH representation to over-darken on the opposite side of the sphere, which can lead to totally black (or even negative!) values.

The other nice thing about SH is that it's really simple to integrate. With specular pre-integration you usually have to integrate for each texel of your output cubemap, which is why importance sampling is used as an optimization. If you're integrating to SH you don't need to do this, you're effectively integrating for a single point. This means you can just loop over all of the texels in your source cubemap, which means you won't "miss" any details. You can look at my code for an example, if you want. Edited by MJP

Share this post


Link to post
Share on other sites

@Radikalizm (Hey! Leuke verrassing hahaha :D ) I was indeed using the same approach. Glad I had the correct idea about why it was going wrong. You also confirmed my hunch about that I might solve it by using a very small, downsampled map (I kinda had that working already, but didn't believe it was a decent approach). The LUT was indeed there for the fancy view dependency as I am trying to replicate the rendering of some (artist) tools that seem to do this as well (as far as my eye can tell); Substance Designer for example. Guess I'll finish up the sampling method by using the downsampled map and then I'll switch over to SH to get a decent comparison between the two.

 

@Krzystof  I even read in a few places that they photoshop the sun out of it to avoid aliasing, but I wanted to keep the ability to load any HDR map robustly. As some tools clearly did this, I started wondering how. On top of that, with the sampling approach it also happened in captured cubemaps with decent values if the difference between values was extreme enough. Thx for the SH link, as well!

 

@MJP Thanks for pointing out the possible edge cases, that will help a fair amount when I run into issues (and I usually do ^^). The code will also be of great help.

Share this post


Link to post
Share on other sites

I don't have time to read all this - sorry for that - but my solution (as I was in the exact same situation) was indeed to use a higher mipmap lod bias. I don't think it's hacky, because the lower your sample count, the broader the cone that approximates a given projection area of the cubemap...so the higher the mipmap level. The core of realtime importance sampling. How many diffuse samples do you actually take right now? I made the experience that around 12 should be enough. You can even evaluate this in quarter screen resolution, you know, low frequency light.

Share this post


Link to post
Share on other sites

This topic is 677 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this