Sign in to follow this  

Length in hlsl (confused)

This topic is 2571 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm very confused with the length function in hlsl. From what i can understand it gives me the length of a vector so for example for:

length(float2(4.0, 3.0))

It should calculate the square root of 4^2 + 3^2 = 16 + 9 = 25 i.e. 5

I'm obtaining 4... what the helll ????

Here is the shader that I have used in Fx Composer 2.5. I have debugged it and confirmed that for one of the pixels the value is indeed 4.0


float4x4 WorldViewProj : WorldViewProjection;

float4 mainVS(float3 pos : POSITION) : POSITION{
return mul(float4(pos.xyz, 1.0), WorldViewProj);
}

float4 mainPS() : COLOR {
float a = length(float2(4.0, 3.0));

return float4(1.0, 0.0, 0.0, 1.0);
}

technique technique0 {
pass p0 {
CullMode = None;
VertexShader = compile vs_3_0 mainVS();
PixelShader = compile ps_3_0 mainPS();
}
}


[Edited by - Ivo Leitao on December 1, 2010 12:35:37 PM]

Share this post


Link to post
Share on other sites
It looks like you found weird bug! I took the latest fxc.exe from the june 2010 directx sdk and here the result


float2 cst;

float4 mainps() :COLOR0
{
float a = length(float2(3,4) );
float b = length(cst);
return float4(a.xxx,b);
}





And here the assembly generate


//
// Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111
//
// fxc /Tps_3_0 /Emainps /Cc c:\Demos\test.txt
//
//
// Parameters:
//
// float2 cst;
//
//
// Registers:
//
// Name Reg Size
// ------------ ----- ----
// cst c0 1
//

ps_3_0
def c1, 0, 3, 0, 0
mov r0.x, c1.x
dp2add r0.x, c0, c0, r0.x
rsq r0.x, r0.x
rcp oC0.w, r0.x
mov oC0.xyz, c1.y

// approximately 6 instruction slots used




We clearly see that on cst it compute (x*x+y*y)^0.5 (the square root done by a invert square root followed by the reciprocal)

On the constant, it pick up the first component, x, and updating y did nothing!

I found a sick workaround for you, wrote this like that and the compiler do the math correctly.


float2 cst;

float4 mainps() :COLOR0
{
float2 v = float2(3,4);
float a = length(v);
float b = length(cst);
return float4(a.xxx,b);
}




with the good result:


//
// Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111
//
// fxc /Tps_3_0 /Emainps /Cc c:\Demos\test.txt
//
//
// Parameters:
//
// float2 cst;
//
//
// Registers:
//
// Name Reg Size
// ------------ ----- ----
// cst c0 1
//

ps_3_0
def c1, 0, 5, 0, 0
mov r0.x, c1.x
dp2add r0.x, c0, c0, r0.x
rsq r0.x, r0.x
rcp oC0.w, r0.x
mov oC0.xyz, c1.y

// approximately 6 instruction slots used



galop1n

Share this post


Link to post
Share on other sites
Quote:
Original post by galop1n
It looks like you found weird bug! I took the latest fxc.exe from the june 2010 directx sdk and here the result

*** Source Snippet Removed ***


And here the assembly generate

*** Source Snippet Removed ***

We clearly see that on cst it compute (x*x+y*y)^0.5 (the square root done by a invert square root followed by the reciprocal)

On the constant, it pick up the first component, x, and updating y did nothing!

I found a sick workaround for you, wrote this like that and the compiler do the math correctly.

*** Source Snippet Removed ***

with the good result:

*** Source Snippet Removed ***

galop1n


Tnks a lot for your answer. I have changed my unit tests. I was almost thinking that I was not seeing it clearly :-S

The same happens with other hlsl functions like distance and smoothstep for example.
This is unbelievable :-(. I'm doing a small pixel shader focused language and I have a lot of unit tests, this was one of the first type of tests that I have done. In a professional tool like the Microsoft Hlsl compiler I cannot understand how things simpel as that escape....

[Edited by - Ivo Leitao on December 1, 2010 7:11:01 PM]

Share this post


Link to post
Share on other sites

This topic is 2571 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this