Sign in to follow this  
Numsgil

HLSL equality operator

Recommended Posts

Numsgil    501
I'm getting an odd result in HLSL with equality operators. I'm building a unit testing framework. I have these defines:

[code]
#define CheckEqual(expected, actual) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && ((expected) == (actual)))
#define CheckNotEqual(expected, actual) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && ((expected) != (actual)))
[/code]

And this shader code:
[code]
void Test1()
{
CheckEqual(float4(1,1,1,1), float4(1,0,0,0));
}

void Test2()
{
float4 thing1 = float4(1,1,1,1);
float4 thing2 = float4(1,0,0,0);
CheckNotEqual(thing1, thing2);
}
[/code]

Both tests are giving opposite results of what you would expect. I have a few other unit tests that check a variety of different things, so I'm sure the problem is just within these tests.

As near as I can tell, the equality/inequality operators only check the first elements of vectors. Can someone confirm that that is true? Is it also true for matrices?

Share this post


Link to post
Share on other sites
Hodgman    51328
From the [url="http://msdn.microsoft.com/en-us/library/bb509631(v=vs.85).aspx#Comparison_Operators"]MSDN[/url]:[quote]The comparison operators are: <, >, ==, !=, <=, >=.
Each of these comparisons can be done with any scalar data type. Comparison operators do not support the complex data types such as vector, matrix, or the object types.[/quote]Off the top of my head, I'd try something like:[code]bool VectorNotEqual( float4 a, float4 b )
{
float4 diff = a-b;
return any(diff);
}[/code]
As always with floating-point though, equality tests are [url="http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html"]dangerous[/url], so don't do this in any places where representation errors can have an effect.

Share this post


Link to post
Share on other sites
Numsgil    501
Yeah, I was hoping I could get away with strict equality since it meant I could treat all data types the same and just use ==. But looks like that doesn't work.

Is there a clever way to set things up so that I don't have to know if a data type is a scalar, vector, or matrix? I don't want to have to build unique CheckEqual functions for each possible combination. eg: float2, float3, float4, float1x2, float 1x3, etc. etc.

Share this post


Link to post
Share on other sites
Numsgil    501
Okay, I think I'm pretty close.

I just have this odd issue remaining.

This test passes:
[code]
void Test()
{
float4 diff = (float4(1,1,1,0))-(float4(1,1,1,1));
Check(any(diff));
}
[/code]

Bu this test does not:
[code]
void Test()
{
Check(any((float4(1,1,1,0))-(float4(1,1,1,1))));
}
[/code]

The only difference, as far as I know, is that the first test stores the result of the subtraction in a temporary variable and the second does not.

The Check macro is defined thusly:
[code]
#define Check(isTrue) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && (isTrue))
[/code]

My understanding of HLSL would indicate that these two tests should be the same. I haven't peeked at the assembly yet (that would be the next logical step) but I'm wondering if the issue is obvious and I'm just being dense.

Share this post


Link to post
Share on other sites
Numsgil    501
This is the generated assembly code for the failing test above. It's precomputed the computation (that's great) but it looks like it's calculated it wrong (that's bad). The test that works is storing the result off to registers. and doing the calculation correctly.

[code]
// Not working:

//listing of all techniques and passes with embedded asm listings

technique __unitTestSharpHLSL_PerformTest
{
pass DoTheTest
{
vertexshader =
asm {
//
// Generated by Microsoft (R) D3DX9 Shader Compiler 9.15.779.0000
vs_3_0
def c0, 0, 1, 0, 0
dcl_position v0
dcl_position o0
dcl_color o1
mov o0, v0
mov o1, c0.xxxy // 0,0,0,1 (ie: false)

// approximately 2 instruction slots used
};

pixelshader =
asm {
//
// Generated by Microsoft (R) D3DX9 Shader Compiler 9.15.779.0000
ps_3_0
dcl_color v0
mov oC0, v0

// approximately 1 instruction slot used
};
}
}
[/code]

Any ideas? Looks like the shader compiler is just doing it wrong.

Share this post


Link to post
Share on other sites
Numsgil    501
[code]
bool __unitTestSharpHLSL_isPassing = true;
[/code]

Nothing too fancy, just a global at the top of the function.

Here's the complete HLSL which should be compilable yourself, if you want to give it a try (it's modified slightly, so the new asm is at the end of the code. But same problem). The weird white space is because this is actually a composite of files from a few different places.

[code]
#define CALL_TEST Test

bool __unitTestSharpHLSL_isPassing = true;

#define Check(isTrue) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && (isTrue))
#define CheckTrue(isTrue) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && (isTrue))
#define CheckFalse(isTrue) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && !(isTrue))

#define CheckEqual(expected, actual) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && !any((expected) - (actual)) )
#define CheckNotEqual(expected, actual) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && any((expected) - (actual)) )
void Test()
{
//float4 a = float4(1,1,1,1);
//float4 b = float4(0,0,0,0);
//CheckTrue(any(float4(0, 0, 0, 0)- float4(1,1,1,1)));
//CheckTrue(any(float4(1,1,1,1)-float4(0,0,0,0)));
//CheckTrue(any(b-a));
//CheckTrue(any(a-b));
//CheckNotEqual(a, b);
//CheckNotEqual(b, a);
//CheckEqual(a,a);

//CheckEqual(float4(1, 0, 0, 0), float4(1,1,1,1));
//CheckNotEqual(float4(1,1,1,0), float4(1,1,1,1));

//float4 diff = (float4(1,1,1,0))-(float4(1,1,1,1));
//__unitTestSharpHLSL_isPassing = any(diff);

//__unitTestSharpHLSL_isPassing = true;
//__unitTestSharpHLSL_isPassing = __unitTestSharpHLSL_isPassing && any((float4(0,0,0,0))-(float4(1,1,1,1)));
Check(any((float4(1,1,1,1))-(float4(1,1,1,1))));
//float4 a = float4(1,1,1,0);
//float4 b = float4(1,1,1,1);
//bool isDiff = any(a-b);//(float4(1,1,1,0))-(float4(1,1,1,1)));
//Check(isDiff);


//Check(any(a-b));
}
struct __unitTestSharpHLSL_Vertex_IN
{
float4 Position : POSITION;
};

struct __unitTestSharpHLSL_Vertex_OUT
{
float4 Position : POSITION;
float4 Color : COLOR0;
};

__unitTestSharpHLSL_Vertex_OUT __unitTestSharpHLSL_TestVS(__unitTestSharpHLSL_Vertex_IN vertexIN)
{
__unitTestSharpHLSL_Vertex_OUT returnMe;

__unitTestSharpHLSL_isPassing = true;

returnMe.Position = vertexIN.Position;

CALL_TEST();

if(__unitTestSharpHLSL_isPassing)
{
returnMe.Color = float4(1,1,1,1);
}
else
{
returnMe.Color = float4(0,0,0,0);
}

return returnMe;
}

float4 __unitTestSharpHLSL_TestPS(__unitTestSharpHLSL_Vertex_OUT pixelIN) : COLOR0
{
return pixelIN.Color;
}

technique __unitTestSharpHLSL_PerformTest
{
pass DoTheTest
{
VertexShader = compile vs_3_0 __unitTestSharpHLSL_TestVS();
PixelShader = compile ps_3_0 __unitTestSharpHLSL_TestPS();
}
}

//listing of all techniques and passes with embedded asm listings

technique __unitTestSharpHLSL_PerformTest
{
pass DoTheTest
{
vertexshader =
asm {
//
// Generated by Microsoft (R) D3DX9 Shader Compiler 9.15.779.0000
vs_3_0
def c0, 0, 0, 0, 0
dcl_position v0
dcl_position o0
dcl_color o1
mov o0, v0
mov o1, c0.x

// approximately 2 instruction slots used
};

pixelshader =
asm {
//
// Generated by Microsoft (R) D3DX9 Shader Compiler 9.15.779.0000
ps_3_0
dcl_color v0
mov oC0, v0

// approximately 1 instruction slot used
};
}
}
[/code]

Share this post


Link to post
Share on other sites
Numsgil    501
I confirmed this as a compiler bug with Microsoft.

The fixed version of CheckEqual uses a dummy variable to trick the compiler into using a different code path:
[code]
static bool __unitTestSharpHLSL_isPassing;
static float __unitTestSharpHLSL_dummy = 0; // to work around a compiler bug in HLSL

#define CheckEqual(expected, actual) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && !any((expected) - (actual) - __unitTestSharpHLSL_dummy) )
#define CheckNotEqual(expected, actual) __unitTestSharpHLSL_isPassing = (__unitTestSharpHLSL_isPassing && any((expected) - (actual) - __unitTestSharpHLSL_dummy) )
[/code]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this