I'm trying to point out the argument cuts both ways. You can't say that "C# is slow by design" and ignore that the de-facto "high-performance" language (C++) also has slow features (i.e. virtual dispatch).
I would argue: "Everything on the heap" is at the core of C#. Virtual dispatch is not at the core of C++ in the same way.
As I already pointed out - C# does not require the heap for everything, and garbage collected (or at least loosely-pointed) memory has it's own advantages (memory compaction).
It doesn't, but then you're fighting the language. Trying to manage an array of value types comes with significant limitations in C#. True though, memory compaction is a potential advantage. And in practice, if you use an array of reference types and pre-allocate all the objects at the same time, they tend to end up sequential in heap memory anyway - so that does mitigate some of the cache performance issues.