Sign in to follow this  

XMVECTOR comparisons won't work in debug mode

This topic is 1780 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

EDIT:  I didn't mean to post this three times.  It gave me an error the first two tries and I didn't realize it went through anyway.

 

XMVECTOR comparisons such as XMVectorNearEqual or XMVectorGreater are not working in debug mode, but they work in release mode.
 
An example:
 
 


 

 

 

    XMVECTOR test1 = XMVectorSet(1, 1, 1, 1);
    XMVECTOR test2 = XMVectorSet(1, 1, 1, 1);
    XMFLOAT4 nearEqualTest;
    XMStoreFloat4(&nearEqualTest, XMVectorNearEqual(test1, test2, XMVectorReplicate(.01f)));
    if(nearEqualTest.x == 0xFFFFFFFF)
    {
        exit(5);
    }
 

 


 

This code correctly exits in release mode, but not debug.  It shows up as -1.QNAN 

 

For reference, this is the function documentation

http://msdn.microsoft.com/en-us/library/windows/desktop/microsoft.directx_sdk.comparison.xmvectornearequal(v=vs.85).aspx

Edited by Vexal

Share this post


Link to post
Share on other sites

That function sets the bits of each component to the integer value of 0xFFFFFFFF. Your "if(nearEqualTest.x == 0xFFFFFFFF)" equality test will promote 0xFFFFFFFF to a float, which won't be the same value as integer 0xFFFFFFFF being reinterpreted as a float (the bits 0xFFFFFFFF actually don't form a valid floating point number, which is why you get -1.QNAN in the debugger). The reason 0xFFFFFFFF is used as the result is so that it can be used as a mask for subsequent SIMD mask operations that choose a particular value based on your conditional. DirectXMath exposes this functionality using XMVectorSelect.

 

If in doubt you should always look at the implementation of DirectXMath functions, since they're all inline in a the .h/.inl files.
 

Share this post


Link to post
Share on other sites
Sign in to follow this