XMVECTOR comparisons won't work in debug mode

Started by
1 comment, last by MJP 11 years, 1 month ago

EDIT: I didn't mean to post this three times. It gave me an error the first two tries and I didn't realize it went through anyway.

XMVECTOR comparisons such as XMVectorNearEqual or XMVectorGreater are not working in debug mode, but they work in release mode.

An example:



    XMVECTOR test1 = XMVectorSet(1, 1, 1, 1);
    XMVECTOR test2 = XMVectorSet(1, 1, 1, 1);
    XMFLOAT4 nearEqualTest;
    XMStoreFloat4(&nearEqualTest, XMVectorNearEqual(test1, test2, XMVectorReplicate(.01f)));
    if(nearEqualTest.x == 0xFFFFFFFF)
    {
        exit(5);
    }
 


This code correctly exits in release mode, but not debug. It shows up as -1.QNAN

For reference, this is the function documentation

http://msdn.microsoft.com/en-us/library/windows/desktop/microsoft.directx_sdk.comparison.xmvectornearequal(v=vs.85).aspx

Advertisement

Do you use version 2.4 or newer?

2.3 had a few bugs.

That function sets the bits of each component to the integer value of 0xFFFFFFFF. Your "if(nearEqualTest.x == 0xFFFFFFFF)" equality test will promote 0xFFFFFFFF to a float, which won't be the same value as integer 0xFFFFFFFF being reinterpreted as a float (the bits 0xFFFFFFFF actually don't form a valid floating point number, which is why you get -1.QNAN in the debugger). The reason 0xFFFFFFFF is used as the result is so that it can be used as a mask for subsequent SIMD mask operations that choose a particular value based on your conditional. DirectXMath exposes this functionality using XMVectorSelect.

If in doubt you should always look at the implementation of DirectXMath functions, since they're all inline in a the .h/.inl files.

This topic is closed to new replies.

Advertisement