There isn't really a lot of overhead to using a "bigger" server than you need. Yes, it will load a bit more code and use a bit more memory, but if it has features you need, that's probably OK.

There MAY be a cost to using a server that doesn't have a good match to your needs. For example, if you need to scale across more than one server instance, then if your application server assumes that applications are stateless web apps, you will have a poor match to requirements.

One common model of development is to build the servers as intermediate layers, using some kind of messaging system (ZeroMQ, Thrift, GRPC, protocol-buffers-over-TCP, or whatever) and then build a separate front-end/gateway for the particular kinds of communication protocols you need. Thus, you could use a Node.js server with socket.io for translating websockets to your server mechanism, you could use an nginx/PHP gateway to translate polled HTTP to your server, and you could use something in Erlang to translate between mobile phone UDP packets and mobile push. Or whatever other combination you want. You'd define game packets using whatever your RPC mechanism is, and implement that mechanism for the server. Define some way to map "game instance id" to "which server does this live on." Have the gateways know how to map traffic for a particular game instance to the appropriate server, and forward traffic/requests as appropriate. These gateways may also have some kind of semi-persistent store for session authorization state (or use signed tickets/cookies.)

The game servers would, in turn, talk to databases or whatever your persistent store is. This ends up being a four-tier architecture -- client tier, gateway tier, game server tier, and storage tier.

Of course, if you'll only ever use a single server, and don't need scalability, just rolling it all into one is simpler, by far, and costs less to get started, too (no need to spin up and manage multiple different server processes.)

The only general-purpose Java game server I've tried to use was Darkstar, and it ... didn't scale. I don't recommend it.

I didn't quite follow what the advantages would be of using a 4-tier architecture, over a simpler 3-tier one.

The fourth (gateway) tier can do protocol translation for you, so that the back-end only needs to understand one canonical protocol.

Additionally, if you have multiple shards/servers, the gateways can do the dispatch to the correct server for a player connection, no matter which gateway the player happens to connect to.

Yes, this is more complex, because it buys you certain kinds of freedoms and flexibilities, especially if you need protocol translation and if you're using TCP or HTTP connections. On the other hand, if you want the shortest path between client and server, three-tier is smaller and less complex.

If you need to talk to web browsers, and you need a connection that's persistent, then you need to use websockets. Else you should probably use TCP, because it's simpler and needs less supporting libraries. One option is to start with TCP, a 3-tier architecture, and not support web browsers, and when you need that support, you add a fourth tier that translates from websockets to your TCP protocol.

I don't think that's so bad. You typically develop a low-level framing protocol (what's at the beginning/end of a datagram, and how do multiple messages fit in.) That can be tested using unit tests, and then using two processes that just send well-formed empty packets between each other.

Then, you develop a message dispatch and serialization protocol -- for each message within the protocol, how do you know what data type it is, and how do you know who should get it, and once you know the data type, how do you know to unpack it. (And, conversely, how do you pack it.) At this point, you'll typically build some kind of simple reflection system on top of your language, that lest you describe each data type with simple macros or "builder" function calls once, and then marshal to/from the format. Again, this code can be tested using unit tests, and then using simple processes that send/receive messages of designated data types, without putting any gameplay behind it.

Btw, in this phase, some developers try using built-in serialization mechanisms in languages like Java, C#, Python, and so forth, and realize that those mechanisms have totally different design goals than games, and end up generating way too big packets. You will want to define your own packet formats, that rely on both sides agreeing on schema (so field order/types/names aren't needed in the packets) and then implement serialization using reasonably efficient byte packing.

Finally, you tie the gameplay to the messages (entities created/destroyed, players giving commands, entities moving, and so forth,) and at that point, you should be able to build bigger integration tests in your code (poke at the game code to generate the intended kind of events, make sure the right kind of messages are generated into the network code.)

Only after all this is done do you need to set up the full client/server integration test, where you need the "fully formed" (as far as you've gotten) client and server to talk to each other.

I am going to answer specifically for Java & Websockets. I can't tell you whether this is the best technology choice or not for your game that's entirely up to you.

There is no correct choice here but there are a variety of solid options available to you. And by that I mean they're tried, tested, well(ish) documented and have active communities using them so you can ask questions online and reasonably expect someone out there will know the answer.

This is not an exhaustive list mind you. Just some of the more common options to help you evaluate:

• Use a full-blown JavaEE appserver like Glassfish, Wildfly, TomEE, Weblogic, Websphere. This is the enterprise way to go, you have everything and the kitchen sink available to you. This is particularly useful if you think that you'll want to use other JEE standards for the purposes of managing logic & state, transactions, databases, etc.
• Use a JEE servlet container like Tomcat or Jetty - Good if you are only interested in the web subsets of the JEE (like servlets & websockets). These can also be embedded so that your application can run standalone. Here's an example I found of Jetty embedded as a websocket server. 
• Use a JEE server generator like Wildfly Swarm - This lets you pick and choose just the bits of the JEE that you need and lets you link them into your application so you can package the whole thing up into a single jar. Allowing for more light-weight servers when you only need a few JEE dependencies.
• Use the JSR 356 (websockets) reference implementation - If you are simply only interested in running a websocket server using Java's websocket API but without any other JEE gumpf then this is how to do it.
• Use the Play Framework - This is not JEE, it's not 'standard'. It's a Java & Scala web framework with a fresh and modern air to it. This is probably the newest player to the field.
• Use Spring Boot - Spring has always been the main alternative to JEE. JEE has come along way in recent years but historically where JEE has been lacking features or was complex to use then Spring was easier, cleaner and more feature rich.  You can use the Spring Boot websocket starter to get up and running quickly.