I am the author of MathGeoLib. While it is not a library that aims to be a direct plug-and-play solution (I don't actively maintain build systems for different platforms for it), you can try to see if it's something of use.
When I was pondering on whether I wanted to undertake the project of writing MathGeoLib, I did some research on some existing libraries. If MathGeoLib is not suitable, see the Alternatives page in MathGeoLib docs, that lists what I think are the most commonly used math libraries for games.
Chek http://glm.g-truc.net/. It's a really nice library with tones of util functions, and ofcorse vectors, matrix, quaternions... It has nearly the same sintax as GLSL and you don't need to link with your project, just include it.