C++ is usually used when performance is more important.
Not only that but both languages use a garbage collector to pick up and delete unused objects as well - which also uses process time.
When I read about why C++ is faster than higher language X, I always hear Garbage Collectors. I don't know why it is such a common misconception. While I do agree with you in general, I want to look a bit closer at the details:
Garbage Collectors do not make the program slower, quite the contrary. If you use a GC or not, you still have the same amount of memory to clean up. The biggest difference here is that C++ cleans it bit by bit, every time an object isn't referenced anymore, while a GC does it in one fellow swoop.
Garbage Collection has it's downsides, like you said when it kicks in, it tends to stop the whole program for a bit until it's finished, which isn't very nice. It also needs more memory. The Java GC can throw an OutOfMemoryException while a third of it's heap is technically free.
Where a GC is better however is throughput, a program tends to spend less time cleaning it's memory with a GC than with reference counting. It finds cyclic dependancies on it's own, throughput tends to scale with the amount of memory you give plus object allocation is faster than in C++. A modern garbage collector shines when you have a program that has many objects that become garbage quickly and a hand full of objects that stay alive during the whole runtime of the program.
So no, a Garbage Collector does not in general negatively impact performance. It all comes down to what task there is to do. And that's where I agree again . I wouldn't write a high performance math library in Java, nor would I implement a high performance streaming service in C++.
E: also with today's runtime optimization of JVM/CLR, you don't necessarily get a performance advantage when you just switch to C++. You can be quite a bit faster, but you have to know what can be optimized and how.