Languages are tools, a programmer is going to have to be at least lightly familiar with quite a few languages, and its a good experience to know multiple indepth anyway.

I'm not sure why people actually suggest Java for game development, realistically there just is not much of a market of Java games out there and I almost feel like solely blaming Minecraft for the surge in people targetting Java as a games platform. It's seen some light for applet games but it is usually completely crushed by alternatives like Flash in that regard, and as a desktop games program it just has a lot of practical negatives.

Personally I like to recommend C# since it is similar to Java in a lot of ways but has a lot more game support, XNA being a great example. C++ is the de-facto if you want to get into AAA big studio development of course, for different reasons.

I guess my response comes down to: is learning Java bad? Of course not, every language teaches you new things. Should you stick with just Java and bust your butt to get a game dev job using just Java? Honestly, no, you shouldn't. In fact I wouldn't even bother unless you actually find an offer to get such a job using Java. Its worth being prepared to use different languages, especially in a competitive field like game development.