In your original post you asked about compatibility issues between GLM and Direct3D. If you're going to be using Direct3D for your graphics, just use DirectXMath for your math library.
DirectXMath is the public version of XNA Math - the math library that's part of the Xbox 360 Dev Kit (XDK) used by game companies to develop Xbox 360 games. It already utilizes SSE and SSE2 instructions and is about as fast as you can get.
Cheers and good luck!