Templates are a major reason. As pretty much all compilers can't separate template declaration from definition all code is inside the header file.
All these cmath, string, etc. are still regular headers, just without extension. At least VS automatically adds references to the C++ runtime libs, so all you need to do is to include the headers. The libs are imported "under the hood".