I forgot what programmer used to say that picking a programming language is like picking a bar to hang out in. I tend to agree
If you're trying to get laid, the choice between gay and straight bar will matter a lot.
So yes, it matters.
As for Java vs. C++.
They are languages. They cannot do anything. Not even a console application.
So the choice comes down to run-time. If Java:
- JEE, JDK, JME or Android (they are different in the same way C and C++ are different)
If C++:
- The OS (Linux, OSX/iOS, Windows pre-8, Windows 8+)
Each of these is what matters. They determine what can and cannot be done.