Jump to content
  • Advertisement
Sign in to follow this  
BlackMage

SQL as data storage == a must?

This topic is 3667 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I try to transfer an example from my job into game programming: We are a 2-man business and besides a real store, we started maintaining an online-shop about three years ago. In the beginning I had to decide wich system to use - as i am the programmer and maintainer of the project. As we only had a very limited budget and in addition to a narrow product range - I decided to roll something my own using PHP. The first year was quite a mess - but until today the system evolved very much and today we are very happy about the inhouse shop software. Due to the fact that we only have a few products - i decided AGAINST a sql database in the beginning and based the whole software around a custom flat-file-database. I had to integrate sort and maintainance scripts later an - but besides that, it works like a charm. The two biggest advantages are that its really easy to handle and lightning fast. Lets turn this example into gaming and this is where my question arises: Is a SQL database as a form of data storage really a must? Imagine you have a game. Assume its a online browsergame. You have a design doc and wrote down all the components you need. You reach a few hundred objects that need persistent storage - do you think SQL is always faster, better, safer, cooler, more flexible. Or do you think one can have success using a handmade flatfile DB?

Share this post


Link to post
Share on other sites
Advertisement
Quote:
Original post by BlackMage
Do you think SQL is always faster, better, safer, cooler, more flexible. Or do you think one can have success using a handmade flatfile DB?
Given enough time, energy and knowledge, you can almost always build something faster, 'better', safer or more flexible than the existing solutions.

Beating an existing piece of software on all of those counts is a pretty tall order, especially safety - you are trading hundreds of thousands of man hours in development and testing by many developers, for a solution developed and tested by two programmers.

Share this post


Link to post
Share on other sites
Quote:
Original post by BlackMage

As we only had a very limited budget and in addition to a narrow product range - I decided to roll something my own using PHP.


Just a quick business reality check. If someone with adequate experience and competence in this type of project were hired on individual basis, the cost of 1 year project would be well over 100k+, usually 130k+ for individual freelance contract. After taxes and other expenses, this usually results in 60-80k flat income comparable to regular employment.

Or, your limited budget was 130k.

But the real issue is something else. Development of a web sales software today involves a choice of WAMP product and customizing it as needed. The time for initial setup, with entire backup, maintenance and administrative infrastructure, as well as data entry (for < 10k items, excluding additional product presentation or additional processing) should be around 2 weeks for single developer. The total cost should be 4 digits flat, perhaps even 3 digits. WAMP/LAMP development is incredibly cheap most of the time.

Are you sure this was optimal and most cost-effective solution?

Quote:
do you think SQL is always faster, better, safer, cooler, more flexible. Or do you think one can have success using a handmade flatfile DB?


Nothing is ever "faster, better, safer, cooler, more flexible".

The only question is, which is better suited for the problem at hand. Query capabilities have been shown to be very desirable and useful ability. Data integrity as well.

Considering that existing SQL-oriented products provide those, as well as some other features, in addition to whole community supporting it via third-party tools, the choice becomes quite feasible.

If you have no need for queries, then SQL will be suboptimal.

Just make sure to understand the reasons behind a choice.

Share this post


Link to post
Share on other sites
IMHO there's no 'must'. It just depends on where you want to draw the line, and what your needs are.

However, developing your own DB for an online browser game sounds to me like beginning a programming project by first writing your own compiler. Unless I was planning on licensing the compiler for side income, I'd rather just focus on the project.

Share this post


Link to post
Share on other sites
Quote:
Are you sure this was optimal and most cost-effective solution?


Regarding our micro business it was the most effective solution i guess. We serve a niche market, and I mean a real niche market. Our products are so specialised that it actually makes almost no sense selling them via internet. Being a programmer and always searching a challenge - i coded the system in my spare time. So I guess development cost goes down to zero.

Speaking about games...

Quote:
However, developing your own DB for an online browser game sounds to me like beginning a programming project by first writing your own compiler. Unless I was planning on licensing the compiler for side income, I'd rather just focus on the project.


This hits the nail on the head. I think its a time/complexity issue. Of course there is no ultimate wrong/right answer scheme - what counts, is the time you have to invest into a system like this. I think its always a trade-off between efficiency, workload and time. And it seems like everyone has to decide this point on his own - depending on the project he is working on.

What have others done? Any experiences to share? I would like to hear more comments

Share this post


Link to post
Share on other sites
I use whatever is easiest to use and change (which usually involves SQL and/or XML), and don't care about performance until it actually becomes a problem. No point in spending 50% of your time working on a project towards a slightly faster database when it results in only 2% of the overhead.

Share this post


Link to post
Share on other sites
Quote:
Original post by BlackMage
Being a programmer and always searching a challenge - i coded the system in my spare time. So I guess development cost goes down to zero.

Well, no, because it took you quite a few man-hours. Time that you could've spend on other things. :)

Quote:
I think its a time/complexity issue. Of course there is no ultimate wrong/right answer scheme - what counts, is the time you have to invest into a system like this. I think its always a trade-off between efficiency, workload and time. And it seems like everyone has to decide this point on his own - depending on the project he is working on.

I think there's something else to be considered: priorities. Many games have things in common, but some games also have some unique aspects. If your games selling point are realistic visuals, or physics, then it makes sense to devote more time on that than on other aspects. For the more common areas, it's usually better to take an off-the-shelf solution. For those focal areas, it makes sense to build something in-house, to make it stand out from the rest.

Now, storing a few hundred items doesn't sound particularly taxing. A standard SQL installation would likely be more than sufficient. Run a few tests with various database systems and settings if you want and pick the best one, but focus the bulk of your time on the actual gameplay. Make that shine. Nobody's going to care whether there's a custom database or SQL running under the hood. That's not your games selling point.

Share this post


Link to post
Share on other sites
SQL databases aren't really designed to bring you ease of reading files, they are designed to give you optimization in terms of searching, and give your data some measure of fault tolerance and consistency in the face of parallel operations. If you don't do searches over the data by some sort key that doesn't leap straight to the entry you want, or you only hit it with one process at a time, then a flat file is far easier to use and simpler to write.

Multiple users, non-trivial lookups and searches, fault tolerance, storage of huge data sets, distribution of workload: These are what SQL databases bring to the table.

Thus, no, it isn't required in any sense. If your data benefits, then use it. If it doesn't fit the bill, then don't bother trying to shoe-horn it into place.

Share this post


Link to post
Share on other sites
SQL is just that, a Language for performing Structured Queries. Nothing more.

RDBMS is just that. It is a database. It stores data, and maintains relations between data. They are often accessed using SQL.

RDBMS is often implemented with certain guarantees. A common, and quite important one, is ACID property. Based on usage, implementations often provide additonal features, such as load balancing, replication or various domain-specific extensions.

ACID is the most important one. It ensures that if something goes wrong, your storage isn't left in invalid state. This property can be achieved when using files (but rarely is).

SQL interpreter can be written that would operate on files that are not organized as a relational database. Many queries would be slower, but you could use them with all existing SQL products.

Based on experience, sets of products have emerged which pack large number of features. A very popular combinations is LAMP/WAMP (Linux/Windows Apache MySQL PHP). This set reached popularity due to affordability and maturity of individual components.


In most service oriented industries, the more you know about your customers, the better you can operate. Performance may be one factor, but it also might not be. Knowledge comes from your database.

For games, you will want to know when someone logged on, when Joe logged in last time, what is the correlation between abandoned/cancelled accounts and playtime, which NPC do people use most, which least, and what are 3 most popular quest rewards...

So yes, queries are incredibly important (RDBMS). So is reliability (ACID). So are quality products (databases have been perhaps one of first most important systems computers were used for), and these products receive a lot of development time and research.

From technical perspective, relational databases are not suitable for bulk data and streaming processing. For most other tasks, they are mature and perform well, while offering powerful and business- and industry-proven functionality.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!