boost used to be really bad about this, but a while ago, they introduced a layer of indirection between the libraries and the assert functionality. Of course, as I write this not all of boost's libraries actually use it instead of cassert. Fortunately, boost's code is completely open so it's possible to modify the offending headers to use boost/assert.hpp instead of cassert, and BOOST_ASSERT() instead of assert(). This reduces the number of annoyances. Of course a simpler method is just to comment out the assert includes and remember to include your own assert macro before any boost headers.
Then some libraries don't even make that much of an attempt.
It's sufficiently bad that my makefiles eventually all start sprouting cpp -dI rules so I can track down offending header files.