• 10
• 12
• 12
• 14
• 16

Moving Normal Matrix Calculation from CPU to Shader Gives Me A Different Result...?

This topic is 980 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

I was experimenting with some lighting code I had based on the tutorial found here:

http://www.mbsoftworks.sk/index.php?page=tutorials&series=1&tutorial=11

For whatever reason, I decided to move the normal matrix calculation into the vertex shader instead of passing it in via uniform.

Now it looks to me like doing the same thing in the shader is giving me a different lighting result than leaving it in C++ and passing it in. I'm reasonably sure now after repeated checking that I haven't slipped up somewhere with something stupid, so I'm assuming that perhaps I just don't understand how something works (a difference between how GLM and GLSL deal with matrices maybe...? I dunno).

The relevant code from the tutorial source:

glm::mat4 mModelView = cCamera.look();
glm::mat4 mModelToCamera;
spDirectionalLight.setUniform("modelViewMatrix", &mModelView);

mModelToCamera = glm::translate(glm::mat4(1.0), vSunPos);
spDirectionalLight.setUniform("modelViewMatrix", mModelView*mModelToCamera);
spDirectionalLight.setUniform("normalMatrix", glm::transpose(glm::inverse(mModelToCamera)));


So he calculates normalMatrix as the transpose of the inverse of the model matrix, and passes that as a uniform into the vertex shader.

The relevant vertex shader code is:

vec4 vRes = normalMatrix*vec4(inNormal, 0.0);
vNormal = vRes.xyz;


Where vNormal is a vec3 out variable going into the fragment shader.

I copied that, and my C++ code looks like this...

glm::mat4 viewMat = gameCam.look();
glm::mat4 modelToCam = glm::translate( glm::mat4( 1.0f ), buildPos );
modelToCam = glm::scale( modelToCam, glm::vec3( scale, scale, scale * 0.75 ) );

shProg.setUniform( "modelViewMatrix", viewMat*modelToCam );
shProg.setUniform( "normalMatrix", glm::transpose( glm::inverse( modelToCam ) ) );


With similarly a vertex shader that looks like this:

vec4 vRes = normalMatrix * vec4( inNormal, 0.0 );
vNormal = vRes.xyz;


By doing it that way, I get one result, which I assume to be the correct result given how it looks (light colour on 2 sides of a cube, darker colours on the opposite 2 sides, and the top is darker again).

HOWEVER... I get a completely different result (just a solid colour on all visible sides; perhaps suggesting a complete mess of some sort being made of some calculation or other) by doing this in the shader instead:

mat4 normMat = transpose( inverse( modelMatrix ) );
vec4 vRes = normMat * vec4( inNormal, 0.0 );


Where I've obviously inserted the following into the C++ to get the value for the modelMatrix:

shProg.setUniform( "modelMatrix", modelToCam );


Why might this be the case? To me, I appear to have replaced one value calculated in C++ via GLM with another value that I would think to be equivalent, using the transpose() and inverse() functions of GLSL instead.

So I can't figure out what's going wrong... I can't possibly seem to have slipped up anywhere else when the code involved is so tiny and it's just a direct substitution of one term for another, and I've checked it quite a few times for anything I might have missed.

Does GLSL work differently to GLM somehow when dealing with matrices (though I can't see why...)?

I thought at one point it could have been something about each vertex having different values for something or other but that can't be it when modelMatrix is a uniform too and it's set right before the original calculation method's normalMatrix so they're definitely not using different values for the model matrix as far as I can see.

I could of course just leave it be calculated in C++, which is probably the better idea, but I was just curious and decided to experiment and now this is really bothering me...

Share on other sites
Nothing seemed off, so I downloaded the tutorial, made the changes you made, and it works fine. Did you change all instances of "shProg.setUniform"? There are three in the code.

Share on other sites

What's the logic behind:

outPosition = modelMatrix * vec4(inPosition, 1.0);
outNormal   = transpose( inverse( modelMatrix ) ) * vec4(inNormal, 0.0);

Why not just:

outPosition = modelMatrix * vec4(inPosition, 1.0);
outNormal   = modelMatrix * vec4(inNormal, 0.0);

The 0.0 in the normal's w component means that it's only affected by the 3x3 part of the matrix, which usually contains rotation data. The inverse of a 3x3 rotation matrix is the same as it's transpose... So if you're inversing it, and then transposing the result... then you get the same values that you started with! Right?

Edited by Hodgman

Share on other sites

What's the logic behind:

outPosition = modelMatrix * vec4(inPosition, 1.0);
outNormal   = transpose( inverse( modelMatrix ) ) * vec4(inNormal, 0.0);
Why not just:
outPosition = modelMatrix * vec4(inPosition, 1.0);
outNormal   = modelMatrix * vec4(inNormal, 0.0);
The [font='courier new', courier, monospace]0.0[/font] in the normal's [font='courier new', courier, monospace]w[/font] component means that it's only affected by the 3x3 part of the matrix, which usually contains rotation data. The inverse of a 3x3 rotation matrix is the same as it's transpose... So if you're inversing it, and then transposing the result... then you get the same values that you started with! Right?

Yes, the full transpose inverse is only required if shear or (non uniform) scale components are present.

Share on other sites

Nothing seemed off, so I downloaded the tutorial, made the changes you made, and it works fine. Did you change all instances of "shProg.setUniform"? There are three in the code.

Interesting... if it wasn't such a compact piece of code I would assume there was a variable I hadn't set somewhere, or I gave 2 variables the same name or something, but I just can't see what's doing it...

Well I'm just using the idea behind the code in a little side project so it's set up a bit differently and there's only 1 relevant place for the uniform to draw a single cube object. So it's not that I left out setting a uniform.

The 0.0 in the normal's w component means that it's only affected by the 3x3 part of the matrix, which usually contains rotation data. The inverse of a 3x3 rotation matrix is the same as it's transpose... So if you're inversing it, and then transposing the result... then you get the same values that you started with! Right?

Well it comes from the tutorial, where it's used because...

Yes, the full transpose inverse is only required if shear or (non uniform) scale components are present.

Of this.

BTW, does that essentially mean that it's only necessary to use the transpose of the inverse if you're doing a scale like (1.0, 2.0, 1.0)? I assume that's all that's meant by non-uniform scaling... but just want to be sure.

In which case, while it's probably very unlikely in a real application, I am in fact using such scaling in this contrived example.

Either way, it doesn't explain the differing effects that are confounding me...

Out of curiosity I got rid of the transpose and inverse functions, the effect still worked with the original method, then I used the uniform modelMatrix and it came out wrong the same way as it was with the functions...

So whatever is wrong, it seems to be with modelMatrix being passed in wrong somehow... even though the other matrices are being passed fine... hmmm.

This is going to turn out to be something dumb I bet, but I still can't figure it. Couldn't be some sort of compiler weirdness, could it? I did try rebuilding several times, so I don't think so but this is getting weird.

Share on other sites

BTW, does that essentially mean that it's only necessary to use the transpose of the inverse if you're doing a scale like (1.0, 2.0, 1.0)? I assume that's all that's meant by non-uniform scaling... but just want to be sure.

Yes, non-uniform scaling is scaling one axis by one value and the others by another (or every one by a different value)

Share on other sites

Think I've found the problem now... if I have, then it was indeed as dumb as I thought the eventual mistake was going to be .

Just looking now, I believe I was putting the uniform setting code in the wrong, very similar and similarly named function that I was using for an earlier test without lighting... which would mean that the reason it was all a solid colour is that modelView was never being initialized.

Yes... it is in fact working now. *Sigh*. Just as dumb as I thought but the idea never occurred to me because it all seemed to make sense... though if I'd been more careful I might have noticed the lack of setting uniforms that were to be used in the fragment shader which should have tipped me off, but of course I wasn't paying attention to that...

In my semi-defence, I'm sure the setting of the normal matrix was mistakenly in the wrong function that I was looking at, despite it not being used at all, so that made it look more legit... oh well.

Thanks for the help. These are at least the kind of frustrations that help drill into you the importance of being properly thorough and paying attention to every conceivable detail...

Share on other sites

And BTW, is there any disadvantage or reason not to use the transpose of the inverse for normals just in case you end up doing non-uniform scaling? Does it complicate things much or result in noticeable framerate drops in some scenarios?

Share on other sites

And BTW, is there any disadvantage or reason not to use the transpose of the inverse for normals just in case you end up doing non-uniform scaling? Does it complicate things much or result in noticeable framerate drops in some scenarios?

Inverting a matrix is a very expensive CPU operation and should be avoided whenever possible.
However if you are using non-uniform scaling then it is not a case in which it can be reasonably skipped, as the normals will be stretched noticeably inconsistently.

Imagine a sloped object at 45°.
     ?
/|
/ |
/  |
/   |
¯¯¯¯¯¯
The normal is 45° the other way, with the [X,Y] components being [-0.70711,0.70711].
Scale vertically by 2.
Without inverse-transposing the normal’s Y will have become 1.4142 ([-0.70711,1.4142]), the vector becoming [-0.44721,0.89443] after normalization.
The actual normal should be [-0.89443,0.44721], pointing further to the left, vertically decreasing rather than increasing.

L. Spiro Edited by L. Spiro