Archived

This topic is now archived and is closed to further replies.

GergisKhan

MMORPG Data Storage questions.

Recommended Posts

Greeting, all. This is my first post (yeah, yeah, we get that all the time...) and so I''ll spare you the gory details. I have several questions regarding MMORPG development as I am writing one. To give you some background, I am a Java software engineer with about eight years in the middle-ware space. So threading, communication, and databases are not new to me. What IS new is the whole concept of tile engines, sprites (though I do remember sprites from my Commodore 64 days) and what not.... and then there''s the new problem I''m working through. Your help is appreciated. Here''s the problem. I have a game based off of a table-top RPG. The game map is approximately 40 miles by 30 miles, and since the game is usually played with a Gamemaster, only the gross details, like "there is a town here somewhere in this 25 square mile area" is given; the rest is up to the GM. So now here''s the problem. In calculating memory requirements for the server, I can''t see how I can have a data structure in memory, be it a Vector, LinkedList or whatever (still not decided till I figure this out) less than half a GIG of memory. The reason is that I figured out to have any reasonable level of detail I would need to divide each 25 square mile area into 1000 "tiles" by 1000 "tiles". This gives the resolution, if we fudge a mile into 5000 feet, of about 1 "tile" being about 25 feet in each dimension. I''m a bit worried, as the initial projects are about 48 MILLION tiles. How on earth do I store that? Or should I just give up now and start dusting off the SQL books again? Also, while the server has to manage that, does this make sense (25 foot tile) for the client? Oh BOY do I see a long road ahead. Regards and advance thanks, GergisKhan

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
It could depend partly on how many clients you have viewing it simultaneously -- or how clustered they are. You could have a rotating structure, that only loads (from disk) the relevant parts of the map, hopefully cutting down from 25sq miles to a 1 sq mile chunk -- whatever your horizon maxes out at, anyway. It could make for a lot of disk-thrashing tho, and you''d need some intelligent pre-loading algorithms.

Share this post


Link to post
Share on other sites
Well, I had been thinking.

I don''t think a server should be responsible in any way for rendering or display of a map tile. I was only thinking that a client would follow the following idea:


Client: character moving off of display on screen to tile XY.
Server: receives message
Server: lookup tile XY and discover what type of terrain and/or items are here (this is the part where I''m lost... what am I looking at, a database? Memory?)
Server: send to client details on terrain
Client: based on server reply, render terrain from locally stored map tiles, etc.

What do you think, all?

Thanks for replys.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Shouldn''t the client just have a local copy of the map? If the terrain isn''t changing all the time, there''s no reason to send that info back and forth across the net all the time.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
4.8 x 10^7 tiles * 2 bytes / tile ~= 91 megs of map data. Not that large on todays modern computers. Using memory map files or some custom caching scheme you can dynamically load parts of the map into memory which you need, and write out changes if needed too, on a 91 meg file. You can keep it all in memory if you have enough memory, which is not inconviable in todays world of 256 meg machines. Now if you had 4.8 x 10^9 tiles, that might be a problem

Good Luck

-ddn

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
4.8 x 10^7 tiles * 2 bytes / tile ~= 91 megs of map data. Not that large on todays modern computers. Using memory map files or some custom caching scheme you can dynamically load parts of the map into memory which you need, and write out changes if needed too, on a 91 meg file. You can keep it all in memory if you have enough memory, which is not inconviable in todays world of 256 meg machines. Now if you had 4.8 x 10^9 tiles, that might be a problem

Good Luck

-ddn

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
4.8 x 10^7 tiles * 2 bytes / tile ~= 91 megs of map data. Not that large on todays modern computers. Using memory map files or some custom caching scheme you can dynamically load parts of the map into memory which you need, and write out changes if needed too, on a 91 meg file. You can keep it all in memory if you have enough memory, which is not inconviable in todays world of 256 meg machines. Now if you had 4.8 x 10^9 tiles, that might be a problem

Good Luck

-ddn

Share this post


Link to post
Share on other sites
Keeping it in memory is fine... until you need to take down the server for maintenance, or the unthinkable happens and you have a crash.

In that case, one must take care to persist the world into some sort of hard disk storage.

My question: SQL database? Flat file?

Any thoughts here?

GergisKhan

Share this post


Link to post
Share on other sites
I agree with one of the anon posters saying to have the basic map locally on the client. The server has in addition all the dynamic items.. like goodies and the location of NPCs or other stuff. Then I''d separate the landscape in logical tiles and swap them in and out of memory (like mem pages) when they are needed. This doesn''t work very well of course when your clients are evenly spaced on the map. I can''t see the advantage of a SQL database here. If you would have some sort of frontend for the game admin where you could place goodies or quests /whatever then a database might make sense but only to regenerate the map file. How do you keep your game in sync ?

Share this post


Link to post
Share on other sites
Hi,

about now, we are tackling with the same problem. And we are sure that we found the right solution. But the approach we chose is completely different.
In the long run, our goal is to create a completely distributable data/computation model, that is absolutely transparent and we want to have it work on a client/server (maybe servant/commander in an n:m ratio where n is about 10 to 100 times lower) architecture.
Our analysis of the map representation/storage produced a multi layered vector model of the map.
I will not reveal much, because we want to protect our design/implementation details, but to mention ONE specific view of the model ...
The tileable view
Well if the client wants tiles, let''s provide another adapter layer (client side), that sits in between the real view and the mathematical model and handles caching, information retrieval, synchronization and the other needed operations.
This leads to a mixed solution, that may be more computationally intensive, but tends to consume far less memory and thus network bandwidth.

Have a nice day creating all those cool games

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
I was thinking too, but don''t pay too much attention to me as I am new as well.
Having done much in webdesign, I thought a XML approach might be good. As I see it, you only have to relay information as in where what is, not specific pictures. Pictures might be referenced and downloaded as needed (chaching?). As you will definetly have lots of repeats of tiles (there will be lots of grass on a 25 squaremile area, even though there is a town), this might be worth thinking of... I hope...

bye
max

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Hi there!

Im also working on a MMORPG, but in 3D. We are going to use threads for each client, and since both the server and the client has the map, and no major changes are going to happen on the terrain, there is no need to send info. Of course at start, the server as to check if the map is ok, so there will be no cheating. We only load into memory no more than necessary, stuff like the players stats and is current frustrum.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Hi there!

Im also working on a MMORPG, but in 3D. We are going to use threads for each client, and since both the server and the client has the map, and no major changes are going to happen on the terrain, there is no need to send info. Of course at start, the server as to check if the map is ok, so there will be no cheating. We only load into memory no more than necessary, stuff like the players stats and his current frustrum.

Share this post


Link to post
Share on other sites
GergisKhan:
To your original question - you''re solution is correct.. Only the client needs to load the graphical information up - the server just passes on and stores information. It needs to have a form of the map in memory, to check for such things as collision detection etc to make sure clients are not cheating - with a tile engine this can be done as simply as storing a byte for each tile, or whatever you need. Servers these days are not unlikely to contain 512mb - 1gb ram, so there should be no problems there.

To your second question on storage:
Data needs to be preserved in case of a crash. If you use databases or files - it''s up to you, each person users their own system (just look around these boards). In my view, you want to be accessing the hdd as little as possible due to how slow it is. Therefore constantly saving information on the hard disk is not practical. Instead I would suggest storing everything in memory, then periodically saving it to a disk in a seperate low priority thread (yeah, most likely causing a little slow down for a small amount of time). This way, if you crash you can load from the most recent backup which wont be TOO far behind. You might also try saving the file on the applications closing functoion, as even with some crashes, the app can manage to perform a few closing ops.
Unfortunately I don''t have enough information on databases to discuss those. From the sounds of things you have that covered.

In the end it''s all your call

Hope that helps,

Dachande

Share this post


Link to post
Share on other sites
What about saving a change on hard drive as soon as there''s one ?
I.E. I buy that long sword => immediately stored on Hard Drive server.

------
GameDev''er 4 ever.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
WHUT THE F**K ARE YOU ALL DOING?????????

Just store all the data in seperate map files (on the client) that are each like 100 MB, don''l load all the data into the ram, say 100MB holds 25sq m, then load _only_ enuf for gameplay to continue w/out too many read/writes, not memory stuff invloved

create an intelegent loading algorithm, hav any of you played Half-Life or Quake2, it loads after youve finnished a level but the load is seemless except the few seconds its flashing the "loading..." text.

even diablo II uses this method. the only informatin the client would have to receive from the server is where the badies are, and what entities are hanging round eg. daggers, swords, guns, armour, money etc.

you do realise though, your talking about a MUMA size server!
the server applications are basicaly __VERY__ efficient database applications - 30+ HDDs all RAID and REALLY FAST supplying the information to the RAM which is feed of throught the 12 OC3 Internet BackBone Lines =)

Share this post


Link to post
Share on other sites
That''s, in fact, if you are lucky enough to have such a mondo server, which I won''t when this starts online.

To all: you have been VERY helpful. It seems I''m on the right track instinctively, but not in practice and that is what you have provided.

Looks like a 100 MB memory database with a low-priority database writer thread, and the clients handle all the image graphics.

Thanks, everyone!

Joe

Share this post


Link to post
Share on other sites
I've thought about this problem before, and this is how I think MMORPGs handle it, but I'm not sure. If I understand you correctly, the question you are asking is how to handle memory requirements for an extremely large world on the server - memory requirements so large that you can't fit the world onto a single server??

I think the way most MMORPGs handle this problem is split the world up into geographical zones, called, well.. zones. Each zone is handled by a different a different thread or process, I'm not exactly sure how this works - but I'm 95% sure its different processes. Each process may or may not be running on the same server. (This is the way EQ handles things.) Each zone can be on a different server.

When a player moves between one zone(process) and another zone(process), you would have to do things like:

    
// these are the names of zones in EQ

Zone Sebilis; // running on one process

Zone GreaterFaydark; // running in a different process

PC thePlayer;

// suppose the player is in sebilis and wants to move to GreaterFaydark

Sebilis.removePC(thePlayer); // the Sebilis process does this

// the Sebilis process would then send a packet/message/whatever to the GreaterFaydark process telling it that it needs to do something like:

GreaterFaydark.addPCat(thePlayer, position);


The tricky part is handling the load and distributing the processes among the zone servers. Depending on the number of players in each zone and the size (memory requirements) of the zone, you need to distribute the processes among the available servers. There are lots algorithms for this I'm sure. I know that AO uses a system where the load is dynamically balanced among the servers depending on past statistics. During the beta, each time the servers were restarted, it would take a few days for the servers to accumulate statistics and figure out how to rebalance the load, and the GMs would tell us not to play for a while if we didn't like the lag. There are also third party programs out there for redistributing the load from serveral processes among many servers. These programs are typically used at various academic institutions and research facilities to make sure their processing tasks are distributed among all the PCs.

There's still the client side of things. There is the problem of transitions between zones. You want the player to be able to see into the other zone if they are close to the zone boundary. If zones are in different processes, how can one zone get access to the world geometry from the other process?

MMORPGs typically handle this by duplicating part of the terrain of the second zone on the boundaries of the first. Perhaps sometimes you've been playing a MMORPG and you've run through a zone and you didn't zone and found yourself in this area? I know AO had a big problem when you were zoning out of Rome Blue, you'd sometimes not zone and you'd get stuck in this area.

Another trick is to use the terrain to naturally separate the zones. Usually they will have one zone connected to another by a long twisty passage, or a corridor with sharp turns (you'll see this alot in EQ). This way you don't have to duplicate much of the zone geometry.

There is one more client-side issue. The issue of load times. I'm not sure how AO reduces their load times so low compared to EQ. I know they store their world geometry data on the HDD in a read-straight-into-memory format with very little or no compression. I think the solution that most MMORPGs use is that they anticipate the loading and preload the zone when they think the player is going to be zoning soon. You'll notice in most MMORPGs that zoning by running through a zone is a lot faster than zoning if you gate/teleport (this is not true in EQ). When you run, the client has a much larger time to anticipate and start preloading the zone.

If anyone has any comments or things to add, I'd be much interested.

EDIT - I reread your post and I had some more thoughts. I'm not sure how EQ/AO handle the world geometery. If I were a MMORPG designer I would think seriously about not loading the world geometry in the server at all. I could then make the clients responsible for collision detection and such, so that the world geometry only be needed to be loaded on the client. I would still maintain a quad-tree or sphere-tree of mobile objects and their positions on the server for proximity testing. But the big problem with this is security and hacking - I'm not sure what to do, I can think of some pretty big (hacks) abuses here.

[edited by - Z01 on March 30, 2002 1:22:08 PM]

Share this post


Link to post
Share on other sites