View more

View more

View more

Image of the Day Submit

IOTD | Top Screenshots

The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

Sign up now

normalize() producing different results

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

6 replies to this topic

#1george7378  Members

Posted 28 January 2014 - 03:27 PM

I've come across something weird in HLSL. Well, maybe it's not that weird, but it was unexpected for me. There's probably a simple explanation. Anyway...

when I do this in my normal mapping shader:

float3 normal = normalize(2.0f*tex2D(TextureSamplerN, PSIn.TexCoords) - 1.0f);


...I get a different result to if I do this:

float3 normal = 2.0f*tex2D(TextureSamplerN, PSIn.TexCoords) - 1.0f;
normal = normalize(normal);


It seems like the latter one produces the correct result when I run my program, but I have no idea why they're different. Any ideas?

Thanks!

#2Mona2000  Members

Posted 28 January 2014 - 03:44 PM

POPULAR

In the first snippet you normalize a float4 and then throw away the fourth component, while in the second snippet you throw away the fourth component and then normalize.

The result of the first snippet is not unit length (unless the sampled fourth component is exactly 0.5).

#3MJP  Moderators

Posted 28 January 2014 - 03:47 PM

POPULAR

Like Mona mentioned, in both cases you're implicitly truncating a float4 to a float3. This is generally considered bad practice, since it can lead to non-obvious bugs due to dropping data. In fact the compiler will actually warn you when it happens. If your intention is to truncate, I would recommend using a swizzle like this:

float3 normal = normalize(2.0f * tex2D(TextureSamplerN, PSIn.TexCoords).xyz - 1.0f);


This makes the truncation explicit, and in your particular case also prevents the bug that you're encountering.

#4george7378  Members

Posted 28 January 2014 - 04:55 PM

Ahh, so simple Thanks!

#5Bacterius  Members

Posted 28 January 2014 - 09:51 PM

Is it ever meaningful to normalize a float4 anyway? Apart from the perspective division I can't think of a situation where you'd want to do that rather than normalize the underlying float3.

“If I understand the standard right it is legal and safe to do this but the resulting value could be anything.”

#6Hodgman  Moderators

Posted 28 January 2014 - 10:03 PM

Is it ever meaningful to normalize a float4 anyway? Apart from the perspective division I can't think of a situation where you'd want to do that rather than normalize the underlying float3.

For normalizing a vector (or turning a position into a vector), you'd use a float3, for normalizing a quaternion you'd use a float4.

Often in graphics you want to normalize a bunch of weights -- if you've got 4 and your weighting system is based on Euclidean distance, you might find normalizing a float4 handy too

Though usually normalizing weights looks more like weights /= dot(weights, (float4)1)...

#7Bacterius  Members

Posted 28 January 2014 - 10:05 PM

For normalizing a vector (or turning a position into a vector), you'd use a float3, for normalizing a quaternion you'd use a float4.

Often in graphics you want to normalize a bunch of weights -- if you've got 4 and your weighting system is based on Euclidean distance, you might find normalizing a float4 handy too
Though usually normalizing weights looks more like weights /= dot(weights, (float4)1)...

I hadn't thought of quaternions, good point!

“If I understand the standard right it is legal and safe to do this but the resulting value could be anything.”

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.