I am not sure if I should go for c++ or c#, I am proficient in both, but I am not sure which one to pick.
We both know that this is exactly why each language would be fantastic for you. I write the obvious for a reason, as a given, so you could end right there and flip a coin to decide or use both languages, though your .NET experience should influence you.
I have been programming in the .Net platform for about 6/7 years, so I quite like to think of myself as a "decent" programmer, and no matter what C# is one, if not, my preferred language
This alone should weigh heavy in your consideration, of course, because transferring ship in the middle of an ocean is not the wise choice.
My doubts are mostly because I don't want to reinvent the wheel, and I feel that C# lacks a lot of good libraries that are available in C++.
Both C# and C++ have plenty of libraries to take anyone from newbie to AAA popular game developer, so really what you wrote is only true for huge development organizations. You will never be able to take full advantage of those extra C++ highly advanced and specialized libraries until you have a gigantic team working with you and probably far beyond your budget to buy software, tools, and licenses to take advantage of them. If you happen to stay indy, then you will be forced to not worry about the extra C++ highly advanced libraries. For the Indy game developer or small team, either C# or C++ would be wonderful if you are a proficient programmer for making game source code.
The C# advanced libraries are slowing closing the gap with the C++ ones. Since C# is a relatively young programming language coming into maturity with libraries being added every year, the day will likely come when this is not a even the unrealistic concern of some programmers it is today. The .NET Framework which you are familiar in using is also evolving to allow C# to have smooth gameplay even with many assets, the way C++ currently does. I will reveal the bridge between older C# tech like XNA and the future in a few moments.
A fantastic game source code could be made in either C# or C++, likely easier in C# for most programmers, and a game engine of the highly advanced kind with a gigantic team and huge budget would best be served with C++ in most cases, having programmers each focus on a specialty area of the game to take advantage of those libraries - no other way. Does their strategy sound like yours? No - of course it doesn't, unless you have 5 to 20 years of heavy overtime available to focus only on game development. Technology advances likely would make your coding implementations troublesome by the time you finally release a game, so don't worry about the C++ libraries which the big game developers use. The C# libraries will keep you busy and growing for years as indy or small team.
SharpDX is very cross-platform for Windows based systems, one of the most efficient to learn, and has good performance, too. It is an example of one of several major ways to make a game written in C# run well. It is extensible, too. Plug-ins are available for it.
Unity 3D simply rocks! Here is another example of plenty of C# library to take you to AAA game status.
A bunch of C++ written game engines allow you to make a game written in C# or other language(s) which looks great, runs smooth, and is extensible.
Both C# and C++ are used to create game coding of AAA popular games. The C# is generally considered more streamlined and easier, while the highly advanced features of some C++ libraries let the large organization take full advantage of the latest and greatest game tech.
Some successful game developers are using C# for game scripting and C++ in a game engine to support it. Among them, some are using an existing C++ game engine to - like you - avoid reinventing the wheel. Other languages are being used and combined in special areas for game development, too. In the coming years, more C# AAA game engines will appear as the current crop of young C# programmers and game devs gain experience and accomplishments. Just like C, Java, and C++, technology will accelerate in order to implement in C# as C# environment comes into maturity with hardware and other advances. This means more libraries are coming in C#!
The C# has all the libraries you will need to make an AAA popular game. The C++ highly advanced libraries will be there until you need them and also until C# eventually catches them in advanced specialty areas. The .NET Framework momentum will lead to only greater things being available from it in the coming years.
Recommendations for you:
1) Find a game engine which allows you to use your C#, C++, and .NET experience.
2) Use C# for scripting the game, handle customizations of the game engine written in C# or C++, and implement by your .NET experience where you can.