This is actually one of the reasons why I like SFML. It's a very minimal and simple wrapper around several libraries that each do one thing, and provides the basic glue that many people normally rewrite and rewrite.
Generally, I prefer libraries that do only and exactly what I want, in a minimalistic way. They should do it with a minimal impact, with a minimal dependency on other libraries.
Super bloated libraries that try to do everything (Qt is like that* - though it's designed well, it tries to be everything) aren't to my preference either.
(IIRC, Boost is actually a collection of separate libraries that are not hyper-dependent on each other (though there are some dependencies), that mostly do one thing. Many of them are even header-file only. I might be mistaken about that though, I've used Boost very little)
*I like Qt and use it, because it's good with native GUIs and you can get things running quickly, but I don't enjoy how bloated it is.