Well, to be fair, exception safety can be a pain in any language. It's true that in C# you don't have to worry about leaking managed resources due to an exception, but it's still easy to stick your program in an inconsistent state especially if you don't follow DRY.
My problem with C++ as a language for beginners is that beginners normally learn by experimentation. They may read a book or take a class but at the end it's them with a compiler trying things out. The first consequence is that in this beginning stage it's very easy for them to think they've learned something about C++ that's actually incorrect. It's not like as beginners they have the experience to generate full coverage tests for their mini-experiments. Even if they do discover something correct about their own compiler they often mistake implementation specific behavior for behavior of the language as a whole. Heck, I've seen teachers do the same thing (evaluation order springs to mind).
The second consequence is that languages with clean grammars and good errors make for much better beginning languages because they can get logical immediate feedback. The semi-colon placement issue is usually a problem because the errors generated frequently point at the "wrong" line of code. Related to this is times when C++ will compile code into something very different than what was intended. The most vexing parse is one of these, but then you've got temporaries created when people wanted to actually wanted to call the base constructor in a derived constructor body, and similar.
Finally, C++ as a first language seems to discourage developing problem solving skills. Beginners learn that when they get stuck against a brick wall, it's probably the language they're fighting and rush off to get help with the latest syntax error or undefined behavior problem. And for people who learned a different language first, that's probably true 99% of the time; however, for someone who's trying to learn C++ at the same time as learning the basics of programming that may not be true in the majority of cases, but true often enough that running off to get help is positively reinforced.
There are people who have successfully learned to program with C++ as their first language. However, it seems like people who start with a different language and then learn C++ later generally get to the level of a proficient C++ programmer faster, and I've seen plenty of both.
It's basically inherited from C, which was developed in a time when computers were sufficiently low on memory that they literally couldn't fit all the symbols necessary for a complex program in memory all at the same time and still have room for actual code. Programmers would be able to identify which symbols were really needed and include only those headers.
My pet peeve: that whole header file / code file separation. Why does C(++) want me to juggle those two when every other language I've ever been exposed to conveniently uses one file for both?