You could have the network class provide a safe and controlled way for each class to read from the network socket, according to a well-defined protocol. That way only the packet identifier is read, and the network class then delegates the work of parsing the remaining data to whatever class should be parsing it, elegantly. And you can still stream from the socket, instead of having to grab the entire packet at once. In this sense, the network class provides a level of abstraction, which is what you wanted.
If you are concerned about reusability, you should cleanly separate parsing (reading data from the packet) and actual work done. Ideally you should be able to simply strip out the parsing code and still have a working class. I don't think creating a dedicated network/class interface for each class is really going to make things better, it's probably overkill (do you really want to deal with EnemyParser, TerrainPacket, and so on?!)