Java can work very well for game development. But while it is safe against crashes, it is not very "safe" for writing fast code. That said, just like you can avoid certain things in C++ and make your code safe ( use as little raw pointer code as possible), you can make your Java run very fast (cut back on garbage collection by using as few "new" and collections as possible). Java will then run blazing fast (faster than other languages which are not C** ).
That said, many game developers do not like java because it was originally not targeted at games. So you will find that you will need to write your own things even for basic stuff. For example: Performant code for in place linear algebra. That is the real problem with java. The game-related frameworks are under-developed and suck.
On the bright side though: C# is almost a copy of Java. In fact it's something purposefully rolled out after getting sued by "Sun" for infringing on their patents. With a good understanding of Java, it should not take you long to learn C#. The syntax is almost identical. The big advantage is that "XNA" used to be a big thing in game development, and as such there was a big community of people who like to write games, and use C#. So the ecosystem around C# for games is more mature. Also on windows you get to use Visual Studio which is awesome! The down-side is that C# sucks on Linux and Android (Mono historically is not good as pure "C#"). Also, good luck debugging cross-compiled C# code on native devices...
So if you want to make a game, go with Unity. C# should be a synch if you know Java.
That being said, lot's of people tell you not to write engines yourself because you won't finish your game.
While true that finishing your game is unlikely, you do learn a lot of stuff by writing your own low level code. Even if you do not finish your games it can be a very rewarding experience. As long as you are not doing this to earn money, then it is a terrible waste of time
Also, not less important then the engine is the Editor. One would argue that especially in the case of Unity, the editor is better than the engine. While Unity is not the "best" engine (not as performant, advanced, or flexible). It is by far one of the best editors (if not the best). A good editor saves you just as much work as a good engine. In Unity you can build your levels and set behaviours in a graphical environment. I like writing my own engines, and they achieve what I want them to. But when I need to collaborate with an artist, it becomes a nightmare. (Ex: Artist doesn't know how to work with the model-files that I support, Artist wants to try his art in a real level, etc...). So the biggest advantage of Unity's editor is that many "non-programers" on your team allready know how to use it. (artists, level designers, musicians, etc... )