Java is slightly slower than C++ across all platforms, however the rendering of your game is done in the graphics API not Java. For instance, to make a nice fast game on windows you would invoke the DirectX API (yes it's true DirectX is faster on windows than Open GL is. On every other platform Open GL is faster, it's a little known fact that windows since Vista has been actively hindering OpenGL to favor Direct X. The underlying operating system some how impairs the path to the video card for anything that isn't Direct X.)
C# is faster on Windows than Java in most cases but still just slightly slower than C++ in some areas. In some other areas C# is actually faster (namely when making direct hardware calls, but this isn't to be confused with DirectX still having control over your video card. C# will perform faster directly reading hardware such as say a USB input device because of the nature of the two part compilation wherein C# programs self optimize themselves to your specific hardware).
We also have come to the point where computers are fast, even the old ones. It's rare you will find anyone who has anything less than a dual core processor, most game players tend to have at least a 1gig video card and 3 - 4gigs of ram. Your serious players, well you probably couldn't lag their system if you tried. So the minor speed losses on some of the lowest level calculations are never really noticable any more. Sometimes the effective performance is actually faster in these "Safer" languages like Java and C# because they protect you from silly memory leaks and in the case of C# on windows it actually molds itself to the hardware it's running on.
I must point out a VERY VERY important fact about Java and it's multi platform capabilities though. Java on Android is NOT the same as Java on PC. Yes Java is the language of android the the majority of the dependency libraries you will be using (Such as OpenGL) are lite or stripped down versions that are tailored to the specific hardwares of the phone. Not everything you can do on PC will be available on the android, be very careful if you are targeting both and make sure you do your research and test frequently. It is not as simple as clicking "Build for Android" to turn your PC game into a droid game!
All in all, most if not all modern computers and I would even risk saying every single one of your potential players computers are plenty powerful enough to muscle through a 2D game in pretty much any language. The limitation would come in to play with the amount of effects and "moving parts" in your visuals. At this point you want to ensure that you are using either Direct X or Open GL. Open GL is the fastest video card API around but windows will limit it to make Direct X work better. Open GL is plenty fast enough on every platform to run a 2D game at the same visual speed as Direct X. If you intend to go multi platform and you are not using a game engine then go Open GL. If you are using an engine or your a perfectionist like me then you want to make make Direct X builds for windows based os' and Open GL builds for everything else. Be aware though, the human eye only see's things at around 33 - 36fps. So when you have Open GL running 90fps on windows and you have Direct X running at 110fps on windows the player will never notice either one being faster no matter how hard they look. (That same game may run at 130fps using Open GL on Mac or Linux but again it is not noticable to the human eye ball).
In short if your comftorable with Java already continue to learn young padiwan, just be aware that your not going to make an MMO out of it. Stick to 2D games and moderate 3D games you'll be fine.