In a typical game, you load the same data on client and server sides, but you "stub out" the graphics-related subsystems on the server, so they don't actually load/keep the graphics data in memory, and the server doesn't actually spend time rendering. A good way of stubbing is to use dependency injection. In C++, this is done through pure abstract base classes; in Java, it's done through interfaces; etc.
I think you absolutely want to have the "physics" data on the clients, too, so that you can give proper UI cues when the client tries to give commands that would walk through a wall or whatever.
If the server generates "secret" data (like the hand of cards in poker) then that should not be sent to the clients until they are supposed to see the data. This may include things like the actual presence of randomly-spawned mobs, for example. However, the location of spawn points and percent chance of spawning ("map data") that may only be useful on the server, can be loaded on the client without ill effect, if it makes the data format and code simpler. It'll just do nothing on the client.
Before anything thanks to the three of you, I'm just quoting the most "complete" answer.
Well, I've trying to do something like what you said, for example I'm designing my maps in a way I will be able to ignore the graphics related part completely when I'm loading it on the server, and you guys pretty much helped with some stuff I was dubious about. And about the info about spawning stuff and all I'm designing it in a way that I won't have to worry about it on the client side, I will just have to worry about a monster when it's actually there, not before it.