Sign in to follow this  
Ivo Leitao

Length in hlsl (confused)

Recommended Posts

Ivo Leitao    172
I'm very confused with the length function in hlsl. From what i can understand it gives me the length of a vector so for example for:

length(float2(4.0, 3.0))

It should calculate the square root of 4^2 + 3^2 = 16 + 9 = 25 i.e. 5

I'm obtaining 4... what the helll ????

Here is the shader that I have used in Fx Composer 2.5. I have debugged it and confirmed that for one of the pixels the value is indeed 4.0


float4x4 WorldViewProj : WorldViewProjection;

float4 mainVS(float3 pos : POSITION) : POSITION{
return mul(float4(pos.xyz, 1.0), WorldViewProj);
}

float4 mainPS() : COLOR {
float a = length(float2(4.0, 3.0));

return float4(1.0, 0.0, 0.0, 1.0);
}

technique technique0 {
pass p0 {
CullMode = None;
VertexShader = compile vs_3_0 mainVS();
PixelShader = compile ps_3_0 mainPS();
}
}


[Edited by - Ivo Leitao on December 1, 2010 12:35:37 PM]

Share this post


Link to post
Share on other sites
galop1n    937
It looks like you found weird bug! I took the latest fxc.exe from the june 2010 directx sdk and here the result


float2 cst;

float4 mainps() :COLOR0
{
float a = length(float2(3,4) );
float b = length(cst);
return float4(a.xxx,b);
}





And here the assembly generate


//
// Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111
//
// fxc /Tps_3_0 /Emainps /Cc c:\Demos\test.txt
//
//
// Parameters:
//
// float2 cst;
//
//
// Registers:
//
// Name Reg Size
// ------------ ----- ----
// cst c0 1
//

ps_3_0
def c1, 0, 3, 0, 0
mov r0.x, c1.x
dp2add r0.x, c0, c0, r0.x
rsq r0.x, r0.x
rcp oC0.w, r0.x
mov oC0.xyz, c1.y

// approximately 6 instruction slots used




We clearly see that on cst it compute (x*x+y*y)^0.5 (the square root done by a invert square root followed by the reciprocal)

On the constant, it pick up the first component, x, and updating y did nothing!

I found a sick workaround for you, wrote this like that and the compiler do the math correctly.


float2 cst;

float4 mainps() :COLOR0
{
float2 v = float2(3,4);
float a = length(v);
float b = length(cst);
return float4(a.xxx,b);
}




with the good result:


//
// Generated by Microsoft (R) HLSL Shader Compiler 9.29.952.3111
//
// fxc /Tps_3_0 /Emainps /Cc c:\Demos\test.txt
//
//
// Parameters:
//
// float2 cst;
//
//
// Registers:
//
// Name Reg Size
// ------------ ----- ----
// cst c0 1
//

ps_3_0
def c1, 0, 5, 0, 0
mov r0.x, c1.x
dp2add r0.x, c0, c0, r0.x
rsq r0.x, r0.x
rcp oC0.w, r0.x
mov oC0.xyz, c1.y

// approximately 6 instruction slots used



galop1n

Share this post


Link to post
Share on other sites
Ivo Leitao    172
Quote:
Original post by galop1n
It looks like you found weird bug! I took the latest fxc.exe from the june 2010 directx sdk and here the result

*** Source Snippet Removed ***


And here the assembly generate

*** Source Snippet Removed ***

We clearly see that on cst it compute (x*x+y*y)^0.5 (the square root done by a invert square root followed by the reciprocal)

On the constant, it pick up the first component, x, and updating y did nothing!

I found a sick workaround for you, wrote this like that and the compiler do the math correctly.

*** Source Snippet Removed ***

with the good result:

*** Source Snippet Removed ***

galop1n


Tnks a lot for your answer. I have changed my unit tests. I was almost thinking that I was not seeing it clearly :-S

The same happens with other hlsl functions like distance and smoothstep for example.
This is unbelievable :-(. I'm doing a small pixel shader focused language and I have a lot of unit tests, this was one of the first type of tests that I have done. In a professional tool like the Microsoft Hlsl compiler I cannot understand how things simpel as that escape....

[Edited by - Ivo Leitao on December 1, 2010 7:11:01 PM]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this