Well if you're not comfortable with C++ then I doubt you will beat the .NET JIT-compiler at writing fast native code in C++, and with Microsoft brewing some native .NET compiler, the speed difference will probably become a lot smaller very soon. (Like compiled VB6 vs C back in the 90s.)
If you want to use C++ because it's your long-term goal to be proficient at it, this is a great opportunity. Using C++ to be able to eventually port to iOS and Android might also be a good idea. If you just want C++ to save a few CPU cycles then I wouldn't bother unless your project is such that it absolutely needs blazing-fast performance. Truth be told, C++ might be a good starting point, but it is not a panacea to slow code. For example, Ogre 3D, which is pure C++, is sluggish and slow, and Runic Games, who are not even making performance-hungry games, had to mod it to improve performance. I wouldn't be surprised if someone could out-perform Ogre3D using XNA.
Also, I'm not sure I'd recommend BinaryFormatter for saving objects during development. I use it in my game editor for saving stuff and I regret it. Whenever I want to refactor something in my code, it breaks binary serialization and I can't fix it by hand, instead I have to use a SerializationBinder, hack some translation code, then load and save ALL my assets, and finally recompile my app without the hack because SerializationBinder slows loading down to a crawl. (It was such a pain that I had to make a library for auto-generating a SerializationBinder.) If I had to code it from scratch again, I'd probably save my levels in XML or JSON and my data in a SQLite database, disk space be damned. I think that binary serialization in .NET is useful for in-process serialization like copy and paste, sending data between AppDomain, and over a network, but not for long-term storage, and especially not during development where stuff can be moved and renamed.