• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
Numsgil

MMO and databases

10 posts in this topic

Are there any books or article series discussing database use in MMOs? For instance, there's a few [url="http://gamedev.stackexchange.com/questions/2282/what-kind-of-databases-are-usually-used-in-an-mmorpg"]one-off posts[/url] on various sites, but I'd be interested in a more comprehensive examination.

I've done a bit of SQL work, and my impression is that it would be a bit like a square-peg-in-a-round-hole situation trying to get the relational model to make sense for an MMO. If we take the simplified case of just inventory management, would you maintain a table of every item in the game universe, and have a field defining a UID for which container it lived in? Wouldn't that make a simple query like "what's in my inventory?" take [i]forever[/i]? Many MMOs get around this by having a limited number of slots in your inventory, with few objects existing outside of players' inventories (or in banks) but if you wanted to really open that up and do an inventory system like the late Ultima games, or even something like Skyrim, where the player can have hundreds of unique items, and there are tens of thousands of unique items randomly placed throughout the world, I imagine things quickly get hairy. You'd probably also want a database model that lends itself to spatial queries (what items exist within a 20 meter bubble of some position in the world), which isn't something relational databases do very well.

I feel like there's enough domain-specific knowledge here that there must be a book or article series somewhere, either in getting relational databases to work well, or in some Non-SQL technology.
1

Share this post


Link to post
Share on other sites
I would do the tables like this.

CREATE TABLE items (
ID INT NOT NULL AUTO_INCREMENT,
Name varchar(255) NOT NULL,
other values...,
PRIMARY KEY(ID)
)

CREATE TABLE bag (
ID INT NOT NULL AUTO_INCREMENT,
PlayerID INT NOT NULL,
ItemID INT NOT NULL,
FOREIGN KEY (ItemID) REFERENCES items(ID),
FOREIGN KEY (PlayerID) REFERENCES player(ID),
PRIMARY KEY(ID)
)

Then you can query for players bag "SELECT * FROM bag WHERE PlayerID = 1;" and "SELECT * FROM items WHERE ID IN (31, 26, 63, 42...);" of course the PlayerID in the bag query is the players ID who's bag you want to look at and the items query ID values are the ItemID's from the bag query.
0

Share this post


Link to post
Share on other sites
Here's some links that may help regarding spatial indexing:
[url="http://workshops.opengeo.org/postgis-spatialdbtips/introduction.html"]http://workshops.opengeo.org/postgis-spatialdbtips/introduction.html[/url]
[url="http://technet.microsoft.com/en-us/library/bb964712(SQL.105).aspx"]http://technet.microsoft.com/en-us/library/bb964712(SQL.105).aspx[/url]
[url="http://msdn.microsoft.com/en-us/library/bb964714(v=sql.105).aspx"]http://msdn.microsoft.com/en-us/library/bb964714(v=sql.105).aspx[/url]

As far as inventory, TMKCodes' approach seems fine. In all or most cases inventory is irrelevant for spatial positioning, effectively in a mini-universe. The trick when using a RDMS is the crazy variety of attributes an item or character may have. For example what buffs apply, what resistances things have, etc etc. I don't have a great answer so far. You can have very wide tables with lots of attributes, spin off sub-tables, serialise data in a single column, etc etc. How does it work if you add new features? Nobody wants to modify the schema of a running MMO if they can help it.
0

Share this post


Link to post
Share on other sites
Even though a RDBMS is quite unsuited to this task, RDBMSs have been and still are used for even less suited tasks (and, successfully!). One can always do some design compromises, shard out, or throw more expensive hardware at it. Is it the perfect approach? Surely not, but it's what works.

RDBMS/SQL comes with 3 very major advantages:
1. It's ubiquitous, there are literally millions of experts on the market, bindings for every language, and middleware readily available
2. The respective implemenations have been tested in millions of installations, concepts have been tested millions of times
3. ACID compliance (which is equally important as it is hard to get right) is understood as a natural, basic features in all mainstream implementations, and it has been tested on millions of installations. It will "just work" and unless you explicitly do something very stupid, it won't bite your ass when you don't expect it.

That said, there is nothing wrong at all with no-SQL key/value stores, especially for game data. Edited by samoth
1

Share this post


Link to post
Share on other sites
Thanks for the comments, everyone.

I'm looking at various NoSQL systems right now. They seem to fit closer to the mental model you have for how an MMO works. [url="http://www.mongodb.org/"]MongoDB[/url] especially seems promising. It has spatial queries and indexing, which you'd need if you want to have, say, large numbers of items floating freely in the world (ala Ultima Online, if anyone remembers that game). In the case of storing bags of items, it also has arrays natively supported. So you could have bags represented as documents in a collection (ie: as rows in a table), with items stuffed in to an array inside each bag document (row). It feels very similar to how tables work in Lua, actually. It seems to map pretty well to OOP. The primary downside is that it's not ACID compliant. So transactions (like moving an item from one bag to another) get tricky and you probably have to manage a two-phase commit manually.

...

To give a different usecase example, I'm trying to scope out how technically possible a "living world" MMO would be. For instance, I'm thinking of trees that spread seeds that grow more trees, with players allowed to chop the trees down or manually move seeds around. To handle it in a general way, I'm imagining that all entities that exist in the world can register themselves to run certain scripts at some future date. So on the server side you'd need to periodically query for entities that are due to run scripts and run them. The scripts can update the state of the entity, change behavior based on neighboring entities, spawn new entities, etc., and write the results back to the database. Clients could request spatial queries to get all the entities in a given area that they're playing to find all the visible entities so that the client can display them. The number of dynamic entites in a given area of the world could be quite large.

You'd probably want to cache highly active entities (like animals) in RAM, and only push changes to the database periodically. Or even treat things like animals totally differently from more static entities. But let's ignore that and just call the entire game state recording mechanism the "database".

As a single player game this wouldn't be so unusual, but making it work as an MMO would be quite difficult. I imagine a traditional RDBMS would quickly choke on something like ths.
1

Share this post


Link to post
Share on other sites
As I said before, you do not want to be building that kind of logic in your data persistence layer. That's just bad architecture.

You should have a game server that manages the spatial lookups, the script invocations, and so on. It would only need to go out to the database rarely (maybe every few seconds) to persist state changes to the world, and you'd only ever need to read back from the data layer when starting up a new server instance.
0

Share this post


Link to post
Share on other sites
Sure, you can do that. But doesn't that mean you basically built your own RAM-only database technology with spatial queries, etc.? Which has all the benefits and pitfalls that rolling-your-own-solution does.

If we assume that "the database" lives across multiple boxes, and the servers running the game logic/script updates are yet different machines, you've basically built a local database cache in RAM on each of the game logic servers, that communicate changes to "the database" lazily. Which sounds a lot like some sort of eventual consistency/[url="http://en.wikipedia.org/wiki/Multiversion_concurrency_control"]MVCC[/url] database system (maybe not exactly? I'm not sure what vocab word fits here)

So if you're going that route anyway, wouldn't it make sense to actually use some sort of MVCC database system directly? A good database system should be able to handle caching things in RAM just fine, so the primary drawback is that the local database cache has a different application memory space. But I think there are some technologies that you can integrate directly in to your application (I think BerkeleyDB can do this?)
0

Share this post


Link to post
Share on other sites
If all your "game" is is a giant database, then sure, you could do that.

As soon as you have to integrate [i]actual game logic[/i] with this separate data system, you [b]will[/b] hurt in the long run. The problem with externalizing all your storage is predominantly access latency. Unless every game server is also a database shard server (waste of hardware) you will have to hit the network to grab every single piece of data you want. Now you have an unreliability concern and a latency problem - expect on the order of a couple milliseconds for every data fetch instead of microseconds at worst. Plus, if your network flakes or a database server goes down, you're screwed; in a persistence-only model, you just keep on chugging and hit the data store as soon as it becomes available again.


Building scalable, performant systems is often counterintuitive. "Rolling your own" may be a sin when you're talking about linked list classes, but when it comes to building an MMO, you [b]need[/b] to have very domain-specific solutions. Integrating with a data store back-end is fine, but building everything around off-the-shelf middleware is a surefire way to kill yourself as soon as you hit scale.
2

Share this post


Link to post
Share on other sites
If you choose MongoDB, make VERY SURE you understand the data persistency guarantees (or, rather, non-guarantees) you get. Also, make sure you understand how to make consistent backups of MongoDB and are willing to go through with such a system.
Personally, I like (and use) [url=http://redis.io/]Redis[/url] a lot for semi-structured-data stores.
1

Share this post


Link to post
Share on other sites
You might also want to consider some new solutions made by [url="http://paralleluniverse.co"]Parallel Universe[/url] (my company). We provide an embedded, in-memory spatial database called [url="http://paralleluniverse.co/product"]SpaceBase[/url], that also helps you parallelize game code, and an open-source data distribution layer called [url="http://puniverse.github.com/galaxy/"]Galaxy[/url]. These solution have been built specifically for MMOs (and some other, similar applications) and work at extremely low latencies. They are not as complete as some NoSQL databases, but that is precisely because we realize, as ApochPiQ said, that to handle specific game requirements there will be some problems you will need to solve yourself with ad-hoc solutions, but we try to help with the really hard general problems. Edited by pronpu
1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0