Sign in to follow this  

MMO (Flat-File)? Databases

This topic is 1963 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I was wondering if any of these ideas could even be possible on hardware today:
Quick overview: A database that will hold probably 1 million+ accounts (probably a overkill in the amount of numbers, but id like to be safe and try to design a system that is easily expandable).
The database will probably all contain, for example, the following...
a) personal information (username, password aswell)
b) list of characters on the account...
- i) character level
- ii) character inventory
- iii) more game-specific data

Thoughts on creating this database:
1. Design my own database system that has a folder containing the account database. There will be 1 .dat file per account. Ive tried something similar like this before and the folder size becomes really large even though there are not large files containing within it, this probably has something to do with the disk state and NTFS... Although I dont think performance would be affected as the database size increases if this is the case though (directly load the single file, no searching required).
2. Same design as above but instead a maximum of 200 accounts per file (not completely sure how this will work though, by that I mean how to find the correct account between multiple files).
3. One main file, dynamically add/remove accounts from this file (I would assume though searching for a account in this solution will have the same problems that SQL databases yield).
4. Just go with technologies such as SQL (big databases become slow though).

The question if i did not clarify: Any of the suggested ideas above even feasibly possible?

As always, thank you. All replies are appriciated.
Xanather. Edited by Xanather

Share this post


Link to post
Share on other sites
A single database table with a hundred million records is not a problem. Storage is cheap and easy to manage.

The question is how many queries per second you want to run on that table. With proper SSDs and a 10:1 read/write distribution, and using read slaves for the read queries, I'd expect you could do 1,000 writes and 10,000 reads per second to a SQL database like that. Note that read slaves are very easy to set up for SQL databases, but much harder to set up for file system based solutions.

You could also put all the accounts in a single Redis instance. I bet you could do several hundred thousand transactions per second there, with the trade-off that if the system crashed, you might roll back to the last disk snapshot (20 minutes or so for a full system.) Redis also supports replicas that you can use as read slaves.

So, to better answer your question about feasibility: The number of accounts doesn't matter much. The number and complexity of queries per second is much more important. Do you have a feeling for what you'd want to accomplish there? Edited by hplus0603

Share this post


Link to post
Share on other sites
[quote name='Xanather' timestamp='1343380631' post='4963572']
1. Design my own database system that has a folder containing the account database. There will be 1 .dat file per account. Ive tried something similar like this before and the folder size becomes really large even though there are not large files containing within it, this probably has something to do with the disk state and NTFS... Although I dont think performance would be affected as the database size increases if this is the case though (directly load the single file, no searching required).
2. Same design as above but instead a maximum of 200 accounts per file (not completely sure how this will work though, by that I mean how to find the correct account between multiple files).
3. One main file, dynamically add/remove accounts from this file (I would assume though searching for a account in this solution will have the same problems that SQL databases yield).
4. Just go with technologies such as SQL (big databases become slow though).
[/quote]
1,2,3 : do this only if you want to write your own DB for fun, not if you need a scalable, working, fast data base.
4: this is the way to go, 1 mio. entiries and a handful of tables are not really a problem. With the right indicies you will have fast access times, the question is, how often will you access the db as hpplus already explained.

From the data you describes access will be most likely not happen very often (login,logout, 5 minut autosave), in this case a simple mysql data base will be likely sufficient and can be scaled up later once you need to support more accounts.

Share this post


Link to post
Share on other sites
Id probably predict... hmm... 2-4? queries per second if I design it well (may probably increase much more though, variates greatly depending on current amount of players obviously). Does this sound feasible for SQL? Edited by Xanather

Share this post


Link to post
Share on other sites
Here's a hint: This site uses a SQL database. It has 4,362,189 Total Posts with 1857 users online (including guests).

It has to sort through all of those posts every time anyone views any thread. Post edits and new posts are written to the database pretty often. Does the site seem responsive? It does to me. I think a SQL database can handle a quarter of that with only a few queries a second just fine.

In this day and age, the only reason to write database software yourself is as a learning experience. If you're serious about performance, stick with the proven projects dedicated soley to building a fast database engine. Hell, MySQL's pretty damn fast and it's free.

Share this post


Link to post
Share on other sites

This topic is 1963 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this