• Create Account

# In-engine environment map probes questions

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

36 replies to this topic

### #21TiagoCosta  Crossbones+   -  Reputation: 3541

Like
1Likes
Like

Posted 22 January 2013 - 10:07 AM

I still don't get how to correctly combine the reflection color sampled from the cubemap with the diffuse and specular color of my material.
If I just add the reflection to the (diffuse + specular) what I get is that the object is completely reflective and you won't even see any diffuse color at all.
On the other hand if I multiply the reflection color with the specular term and e.g. have a material that is highly glossy, this reflection will hardly be seen at all when the highlight is so small.

So as you explained multiplying by the specular term doesn't work, as expected, because the reflection color has nothing to do with the specular term of a light.
Next lets think about the diffuse term:
Multiplying by the diffuse term doesn't make sense either because what happens if you have a glossy black material?
When you multiply the reflection color with the diffuse color (0,0,0) you will loose the reflection which isn't correct right?

That leaves you with the option to simply add.

If the reflection color is too strong maybe you should be using a lower specular color in the Fresnel calculations...

That's problem I haven't looked into yet, and I guess to have an accurate result some form of raycasting will be needed.

I believe you have can use ambient occlusion (eg SSAO) to approximate occluded pixels.

Maybe I'm just thinking wrong but what I'm trying to achieve is something like this:
or the reflections on the ground here: http://seblagarde.files.wordpress.com/2011/12/locallightingcube.png

Those spheres are metallic so the specular color isn't monochromatic, which changes the color of the reflections since you use the specular color in the Fresnel calculation.

Also I understand that reflections from the env map are basically "specular" so in my engine would I use this reflection color on every object ? And if I wanted an object that is completely reflective from all sides would I consider this to be a separate material type ?

Since the environment map color doesn't depend on any lighting it makes sense that it calculations should be done in a separate shader from the lights.

If you have multiply lights in the scene, make sure that you're not adding the environment map color multiple times to each pixel.
For example, if you have a deferred renderer, since you run the lighting shader once for each light, you need a separate shader to add the environment map color just once.

Check this article, for some info on how to make glossy reflections.

### #22lipsryme  Members   -  Reputation: 1438

Like
0Likes
Like

Posted 22 January 2013 - 02:37 PM

Ok so this is how it looks like for me if I add the environment color to the final lighting:

http://d.pr/i/Mp2J

And this is how it looks without:

http://d.pr/i/XHC4

Is there some little thing I'm missing ?

	float3 reflectionColor = EnvironmentProbe.SampleLevel(LinearSampler, R, mipLevel).rgb;

// Compute Fresnel Term (F) using Schlick's approximation
float NdotV = saturate(dot(N_WS, V_WS));
float3 F_schlick = 0.3f + ((1.0f - 0.3f) * (1.0f - pow(NdotV, 5.0f)));
reflectionColor *= F_schlick;

### #23TiagoCosta  Crossbones+   -  Reputation: 3541

Like
0Likes
Like

Posted 22 January 2013 - 03:41 PM

Is there some little thing I'm missing ?

float3 F_schlick = 0.3f + ((1.0f - 0.3f) * (1.0f - pow(NdotV, 5.0f)));

There's an error in that line...
It should be:

float3 F_schlick = 0.3f + ((1.0f - 0.3f) * pow(1.0f - NdotV, 5.0f));

Post a screenshot after changing this line

### #24lipsryme  Members   -  Reputation: 1438

Like
0Likes
Like

Posted 22 January 2013 - 03:53 PM

Ah stupid me, didn't fix the reflection much though...

The object seems to retain more of its diffuse color now.

http://d.pr/i/opeH

### #25TiagoCosta  Crossbones+   -  Reputation: 3541

Like
0Likes
Like

Posted 22 January 2013 - 04:07 PM

Are you calculating the lighting in linear space (instead of sRGB)?
If so, 0.3 as specular color is probably too high since the specular color of plastic is 0.03 (0.21 in sRGB).
(A list on specular color at slide 43)

Another thing I'm guessing is that you're not calculating the view vector correctly... It seems to be constant because the reflection is as strong in the ground close to the camera as from far away...

Make sure you're calculating it like this:

float3 view = cameraPositon-pixelPosition;

EDIT: Post a screenshot looking down directly at the ground.

### #26lipsryme  Members   -  Reputation: 1438

Like
0Likes
Like

Posted 22 January 2013 - 05:48 PM

You're correct about the fresnel color. Still didn't change anything for the better... I've gone over it again, to make sure the view vector is correct. Tested it with a phong (RdotV) output, which worked. The reflection when moving around the sphere's also looks like it would be correct. It is a distant env map though. It's actually the skybox itself, filtered using cubemap gen. But does that even make any difference ? edit: here's the screenshot when looking down at the ground:

http://d.pr/i/lnEX

Edited by lipsryme, 22 January 2013 - 06:01 PM.

### #27TiagoCosta  Crossbones+   -  Reputation: 3541

Like
0Likes
Like

Posted 22 January 2013 - 06:06 PM

Now I'm pretty sure there's some error in the fresnel calculation...
When looking down at the ground the N.L of most pixels should be almost 1, so the Fresnel would be the specular color of the material. Since the material has a low specular color the sky reflection shouldn't that much visible.

Check the result of the fresnel equation when looking at the ground for some pixels in PIX (or NVIDIA NSight), it should be close to the fresnel color.

Also post the full pixel shader if you can... I'm running out of ideas

EDIT: You changed the screenshot looking at the ground right? It looks correct now.

Edited by TiagoCosta, 22 January 2013 - 06:08 PM.

### #28lipsryme  Members   -  Reputation: 1438

Like
0Likes
Like

Posted 22 January 2013 - 06:14 PM

Yeah sorry, I switched the N and V before for testing when calculating the reflection vector.
Still the ground looks like this (it's just this small part in the middle that doesn't get the reflection):
http://d.pr/i/QXfU

Here are the parts that are important for the env reflection:

	// Calculate Position from linear view space depth
float3 ViewRay = input.ViewRay.xyz;
float depth = DepthTarget.Sample(PointSampler, UV).r;
float linearDepth = ProjectionB / (depth - ProjectionA);
float3 ViewDir = ViewRay * linearDepth; // View space position
float3 V = normalize(ViewDir) * -1.0f;
float3 V_WS = mul(V, (float3x3)InverseView);
V_WS = normalize(V_WS);

// Retrieve Specular Power [2-2048]
float SpecPow = exp2(10.0f * AlbedoTarget.SampleLevel(PointSampler, UV, 0).a + 1.0f);

// Sample reflection from environment probe
float3 R = reflect(N_WS, V_WS);

// Pick MipLevel depending on glossiness
float mipLevel = -1.66096404744368 * log(SpecPow) + 5.5; // log(SpecPow / 2048.0f) / log(0.25f);
float3 reflectionColor = ToLinear(EnvironmentProbe.SampleLevel(LinearSampler, R, mipLevel).rgb).rgb;

// Compute Fresnel Term (F) using Schlick's approximation
float NdotV = dot(N_WS, V_WS);
float3 FresnelReflectance = ToLinear(float3(0.3f, 0.3f, 0.3f));
float3 F_schlick = FresnelReflectance + ((1.0f - FresnelReflectance) * pow(1.0f - NdotV, 5.0f));
reflectionColor *= F_schlick;

// Accumulate results of BRDF and Ambient lighting for regular objects
lighting.rgb = MicrofacetBRDF(N, LightDir, V, DiffuseAlbedo, SpecPow,

lighting.rgb += reflectionColor;

### #29TiagoCosta  Crossbones+   -  Reputation: 3541

Like
0Likes
Like

Posted 22 January 2013 - 06:24 PM

It's starting to look better, I'm wondering why there's no reflection in the ground far away in front of the camera because the fresnel should increase with the distance in that situation... Can you post the shader? If you don't mind... (or send me a link to it via PM if you don't feel like sharing it to everyone)

### #30lipsryme  Members   -  Reputation: 1438

Like
0Likes
Like

Posted 23 January 2013 - 06:55 AM

Okay so here you go:

I've removed some stuff that's irrelevant to make it more clean:

### #31TiagoCosta  Crossbones+   -  Reputation: 3541

Like
1Likes
Like

Posted 23 January 2013 - 08:49 AM

It looks correct. So I'm not sure what the problem might be...

Let's try one more thing:
Remove the normal mapping, so the normal of the ground is (0.0f, 1.0f, 0.0f) on all pixels, and just output the reflection color. So there're no changes in normals, diffuse colors and lighting.

The ground should be black near the camera and the reflection of the environment map should increase with distance around the camera.

P.S: I noticed that you're multiplying the reflection color with the shadow factor, which isn't correct because a pixel might be in shadow from the light but it still receives light from the environment. You might want to multiply it with the ambient occlusion factor though

Edited by TiagoCosta, 23 January 2013 - 08:54 AM.

### #32lipsryme  Members   -  Reputation: 1438

Like
0Likes
Like

Posted 23 January 2013 - 10:13 AM

Multiplying the reflection color with the ambient light (and ambient occlusion) does make it look good, but I guess now its intensity would depend entirely on the ambient light intensity in the scene.
I'm wondering wouldn't it make sense to use the glossiness or specular power of the material as an intensity value for this reflection color ? Basically a material that is highly glossy (let's say = 1.0f) would have strong reflection intensity and a material that is very rough (= 0.2f) would then have very low reflection intensity. Does that make sense, or would it be "physically incorrect" somehow ?
Also would I add the reflection to everything ? Or exclude it on materials like skin ? (not sure I can't remember having seen environmental reflections on human skin before in real life).
And what you said earlier about the cryengine screenshot of those sphere's that is a metallic reflection ? How is it different ? Do I just output specular and ignore diffuse completely ? You're saying the color of the spheres in that screenshot comes from the specular color ?

Here's a screen of how it looks multiplied by the ambient and ambient occlusion:
http://d.pr/i/SuZh

The reflections are very faint now. Which makes perfect sense on the ground but maybe not on the highly glossy sphere on the right.

Edited by lipsryme, 23 January 2013 - 10:24 AM.

### #33CryZe  Members   -  Reputation: 769

Like
1Likes
Like

Posted 24 January 2013 - 02:50 AM

Basically a material that is highly glossy (let's say = 1.0f) would have strong reflection intensity and a material that is very rough (= 0.2f) would then have very low reflection intensity. Does that make sense, or would it be "physically incorrect" somehow ?

A rough material reflects just as much light as a smooth one. The only difference is, that on the rough material the reflections are more scattered into all directions and not as bundled into a single direction. That gives the illusion of a highly smooth material being more reflective, but that's not true, it's just more or less "binary" as to how it reflects light (completely into one direction, nothing into another direction).

The only parameters needed to describe reflection are parameters to describe the materials microfacets orientations and positions (usually a single parameter called "Roughness" or "Glossiness") and the index of refraction of the material. The IOR is completely responsible for as to how much light is reflected from a microfacet which can be calculated using fresnel's equations. The easier and more common way to store this IOR is to calculate how much light would be reflected at microfacet normal incidence (aka specular intensity) and use this later on with Schlick's approximation of fresnel's equations which approximates them without the need for actual IOR values.

Each microfacet is a perfect mirror so light coming from one direction will be reflected to exactly one other direction. So how each microfacet is oriented doesn't affect how much light is reflected, only where it's being reflected to.

So overall it doesn't matter how rough your material is, the only thing that is actually responsible for how much is reflected, is the index of refraction.

You could actually derive a diffuse model from this microfacet model as well:
Simply multiply lambert with the integral over all the microfacet orientations of the percentage of microfacets being oriented into this direction multiplied with the percentage of microfacets actually visible from the light source multiplied with the complement of how much light is reflected from these microfacets (because the non-reflected rest of the light is getting scattered into the material and thus might get scattered out again into random directions as diffuse lighting if it's not absorbed).

Also I'm not quite sure what those numbers are (0.2 being rough and 1.0 being glossy). In Phong and Blinn-Phong you have exponents ranging from 1 to infinity where higher numbers represent a smoother surface and in NDFs like Beckmann or GGX you have factors ranging from 0 to 1 where higher numbers represent a rougher surface. You are either using a pretty weird NDF / BRDF or you misunderstand something.

Also would I add the reflection to everything ? Or exclude it on materials like skin ?

Everything has an index of refraction and thus everything must be reflective unless it has the same index of refraction as air (or whatever material the camera is in). Skin for example is pretty much as reflective as water is (the index of refraction doesn't differ that much), but is pretty rough in comparison to water.

metallic reflection ? How is it different ? Do I just output specular and ignore diffuse completely ? You're saying the color of the spheres in that screenshot comes from the specular color ?

Metals have index of refractions in the complex plane. If you calculate how much light is reflected using fresnel's equations, you get pretty high percentages. These are usually around 95% instead of at about 4% for dielectric materials. Also most metals unlike dielectric materials have highly varying index of refractions in the visible light spectrum. So green light might get reflected less than red light for example. Copper and gold are good examples of materials with varying index of refractions in the visible light spectrum. Also metals are highly absorbtive. The already low amount of light that is not reflected and gets scattered into the material gets absorbed so fast, that pretty much no light gets scattered out again as diffuse lighting. Keep in mind, that there is no difference between subsurface scattering and diffuse lighting.

Edited by CryZe, 24 January 2013 - 04:59 AM.

### #34TiagoCosta  Crossbones+   -  Reputation: 3541

Like
1Likes
Like

Posted 24 January 2013 - 06:15 AM

Multiplying the reflection color with the ambient light (and ambient occlusion) does make it look good, but I guess now its intensity would depend entirely on the ambient light intensity in the scene.

I would simply multiply with the ambient occlusion factor (not the ambient light). The reflections might be to strong because the environment map luminance is much higher than the directional light intensity, and when you multiply the reflection color with the ambient light you scale it back and it starts to look better. Keep in mind that a physically based renderer usually requires some calibrating work to be done.

Also would I add the reflection to everything ? Or exclude it on materials like skin ? (not sure I can't remember having seen environmental reflections on human skin before in real life).

No need for that. Even though every material reflects light, there's no need to waste computation time calculating reflections for rough materials because the reflection won't be noticeable. For movies, there's time to add reflections on every material because you don't need to run at 30/60 fps, games, on the other hand, need to cut on details like that.
Skin is actually a good candidate for environment reflections, especially in bald people at grazing angles, but since its a rough material the reflections are so much "blurred" that it looks like a uniform color.

And what you said earlier about the cryengine screenshot of those sphere's that is a metallic reflection ? How is it different ? Do I just output specular and ignore diffuse completely ? You're saying the color of the spheres in that screenshot comes from the specular color ?

Yes, diffuse color can be ignored in metallic surfaces.
Example: Gold specular color (1.0f, 0.71f, 0.29f)
Even if HalfVector dot Light or View dot Normal = 1 the result of fresnel will be (1.0f, 0.71f, 0.29f).

Here's a screen of how it looks multiplied by the ambient and ambient occlusion:
http://d.pr/i/SuZh

The reflections are very faint now. Which makes perfect sense on the ground but maybe not on the highly glossy sphere on the right.

What specular color are you using in the sphere? You might need to increase it. Especially if the sphere is supposed to be metallic.

Also I'm not quite sure what those numbers are (0.2 being rough and 1.0 being glossy). In Phong and Blinn-Phong you have exponents ranging from 1 to infinity where higher numbers represent a smoother surface and in NDFs like Beckmann or GGX you have factors ranging from 0 to 1 where higher numbers represent a rougher surface. You are either using a pretty weird NDF / BRDF or you misunderstand something.

Its common to store a glossiness value in the range [0,1] in textures and then convert it to specular power using a function like this:
float specPower    = pow(2.0f, 13*glossiness);


Edited by TiagoCosta, 24 January 2013 - 06:20 AM.

### #35lipsryme  Members   -  Reputation: 1438

Like
0Likes
Like

Posted 24 January 2013 - 06:45 AM

So would it be correct to multiply this by the ambient occlusion ? (as I recall ambient occlusion should only ever be added to something?) else you'd get the dark aura look of e.g. Far cry 3

And how about shadows ? Do I multiply or add them to the environment map color ? Or not at all ?

But even with a fresnel reflectance color of 0.028f (for skin) I'm seeing reflections even when not at glancing angle on the skin. Do you think the color of the env map is too bright ? I basically just took my skybox and prefiltered it with amd's cubemap gen. And I'm also already reading the env map as linear.

### #36CryZe  Members   -  Reputation: 769

Like
1Likes
Like

Posted 24 January 2013 - 06:46 AM

No need for that. Even though every material reflects light, there's no need to waste computation time calculating reflections for rough materials because the reflection won't be noticeable.

So rough gold would be completely black? I don't know, roughness doesn't change anything about the reflectivity of the material, just the "blurriness" of the reflection. So on metals or at grazing angles you'll lose an incredible amount of lighting if you simply turn it off for rough materials. If anything I'd approximate rough reflection using an approximation via a diffuse term.

Its common to store a glossiness value in the range [0,1] in textures and then convert it to specular power using a function like this:
float specPower = pow(2.0f, 13*glossiness);

Oh, my bad. I thought he was talking about the specular power used in the BRDF.

So would it be correct to multiply this by the ambient occlusion ?

Ambient occlusion is low-quality shadowing information. Use it on anything non-directional, where you don't have better visibility information. So don't apply it to reflections or any point / spot / directional lights. Just to irradiance or other kinds of ambient terms, like diffuse sky illumination. Basically every source of light which comes from more than just a single direction (this includes area lights) could be multiplied with ambient occlusion. The more it spans the hemisphere over the surface, the more it's suited for ambient occlusion. Environment maps might span the whole hemisphere over the surface, but when used for reflections, only a single direction is used, so you shouldn't apply ambient occlusion to it. If the environment map is used for diffuse image based lighting, samples over the whole hemisphere are used and thus ambient occlusion should be applied. For rough reflections you might want to blend in ambient occlusion to some degree though.

And how about shadows ? Do I multiply or add them to the environment map color ? Or not at all ?

I'm not quite sure which shadows you're talking about. But basically you are "tracing" a reflection ray and the environment map helps you find the intersection. So there's no need for any kind of shadow map.

But even with a fresnel reflectance color of 0.028f (for skin) I'm seeing reflections even when not at glancing angle on the skin. Do you think the color of the env map is too bright ?

Either it's actually too strong or the reflection is not rough enough. Hard to tell without seeing the result.

Edited by CryZe, 24 January 2013 - 08:17 AM.

### #37TiagoCosta  Crossbones+   -  Reputation: 3541

Like
0Likes
Like

Posted 24 January 2013 - 04:25 PM

Do you think the color of the env map is too bright ? I basically just took my skybox and prefiltered it with amd's cubemap gen. And I'm also already reading the env map as linear.

Maybe... You have to make sure that intensity of the env map matches the light sources in the scene...

So rough gold would be completely black? I don't know, roughness doesn't change anything about the reflectivity of the material, just the "blurriness" of the reflection. So on metals or at grazing angles you'll lose an incredible amount of lighting if you simply turn it off for rough materials. If anything I'd approximate rough reflection using an approximation via a diffuse term.

You're right, for metals the reflections shouldn't be disabled. I was talking about materials with both low specular colors and glossiness where the reflections aren't much noticeable, like the trunk of a tree, etc.

Ambient occlusion is low-quality shadowing information. Use it on anything non-directional, where you don't have better visibility information. So don't apply it to reflections or any point / spot / directional lights. Just to irradiance or other kinds of ambient terms, like diffuse sky illumination. Basically every source of light which comes from more than just a single direction (this includes area lights) could be multiplied with ambient occlusion. The more it spans the hemisphere over the surface, the more it's suited for ambient occlusion. Environment maps might span the whole hemisphere over the surface, but when used for reflections, only a single direction is used, so you shouldn't apply ambient occlusion to it. If the environment map is used for diffuse image based lighting, samples over the whole hemisphere are used and thus ambient occlusion should be applied. For rough reflections you might want to blend in ambient occlusion to some degree though.

Since the OP is using the same environment map for all objects multiplying the reflections with the ambient occlusion factor might help reduce artifacts like reflections of the environment on the bottom of the spheres near the ground where they should be occluded. This is just a suggestion though, since occlusion in environment mapping is a rather complex problem.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

PARTNERS