Data Driven Design: Questions (and Answers?)

Started by
9 comments, last by vipar 10 years, 9 months ago

I've been reading up on a lot of articles which covers the subject of Data Driven Design (DDD) when making games.

It seems to come down to:

  • No Hard Coding
  • No Game Specific Code in Engine
  • Scripting (ai, cut scenes, etc.)
  • Generalizing Code for Max Reusability
  • Component Design
  • Modularity
  • Low Coupling
  • Editors (Data, Map, Scripting)
  • External Data Retrieval
  • Constants kept in Text Files (INI or otherwise)
  • Expose Data through Editors for Scripting/Manipulation for Designers

Now, my first question is of course: Is this understanding correct? (From an Objective Point of View if possible)

One of the editors I looked at was Warcraft 3's Map Editor (WC3 ME) since I've had the most experience with it over my years of obsessive gaming. For those who don't know, the WC3 ME is the tool used to make custom maps in WC3. You had the power to pretty much alter anything within the engines limitations be it Units/Abilities/Items (Raw Data Editing like in a Database), Maps (Level Editor) or Triggers (Event Based Scripting Editor). You could also write in Blizzards custom language called JASS to gain absolute control of the language the the Trigger Editor tried to cover in as much detail as possible. The WC3 ME was often considered the "Engine" of the game.

My dream is to make an engine for my games which can be as extensible, modable and easy to configure engine as the WC3 Engine. Other examples from more recent games would be Starcraft 2 and Skyrim both also following DDD. Popular DDD engines are UnrealEngine (UDK) and UnityEngine (Unity). Some would argue Torque, but from what I understand it violates some of the core principles of DDD. Especially with it's heavy realiance on the scripting system that kind of developed into it's own programming language which is not part of DDD.

The problem I seem to have though, is where to start. I can't wrap my mind around how to execute this methodology in code. The engine needs to be very abstract so that I can make many different games with little to no edits in the engine itself. My goal is to first make a simple (at least in theory) 2D tile engine to follow styles such as that from Pokemon or A Link to the Past. Moreso the latter because it uses a tile system but got free movement in all 8 directions.

My second question is: Do you have any points on where to start?

I would use an already existing framework to make the engine and not make my own OpenGL bindings and such. Something like Tao or SFML.

So that is that. I hope we can have some good discussions in this thread and perhaps get together to get resources on these subjects as they are very sparse...from what I have tried to find any way!

CEO of Dynamic Realities

Advertisement

As I said on GDSE, you are massively overcomplicating things. Data-driven design just means you keep your code and your data separate, and you allow the code to be directed by the data instead of the other way around. Simple things like loading configuration information from text files put you on the path to data-driven design.

Your problem appears to be that you are suffering from design paralysis because you're trying to take in the entire scope of everything that could be considered data-driven design at all once, including complete existing implementations, so you can rebuild it yourself. This is bound to result in frustration and failure. You need to start small, and build systems that you need for your games. Reading other people's code can be interesting, but often lacking in a lot of critical detail about the why of various decisions that can easily lead you astray.

Your dream goal is to have an engine on-par with AAA titles. That's great, but the way you're going to get there is not to explore a bunch of code other people have written that leverage data-driven techniques. You're going to get there by writing games. Don't worry about "making the engine very abstract so you can write many different games on it" to start with. You will evolve your code there eventually: for now, focus on smaller goals.

Pretty much the only piece of technology you need in place to start with is a way to actually load structured data from files. You can find great third-party libraries for parsing XML or JSON. For example, I use picojson often in C++. You may optionally want to have a serialization layer that uses the external library to turn data files into C++ structures, but for small projects I would not necessarily build that layer to start with. Wait until you have unburdened yourself of this design paralysis that plagues you right now.

Every time you build a new gameplay feature, take note of the inputs to that feature that control how it behave. Consider what is gating every conditional statement and while loop in that feature's code, and ask yourself "does it make sense that I'd want to tweak or change this?" If the answer is yes, consider how to put it into data.

Focus entirely on that concept for now, and finish a single simple game with it. Then, as you move on to another game, take stock of your existing code and see what is immediately re-usable, and what might be re-usable with some refactoring. Start to build that second game, and refactor the components you need as you need to increase their generality. Additionally, with your next project, pick a new aspect of technology to focus on. First you essentially focused on "loading data from files," so perhaps your second project would be the right time to start adding a more powerful serialization layer between the data and the code (so you're not just passing around XML node pointers or JSON objects into your feature code). Or, perhaps it's time to look into how you could implement a system that automatically reloaded data as it was changed.

By refining your code through projects with limited scope and concrete goals (a simple game), focusing each time on a small set of technology improvements, you will find yourself well on your way to having a reusable engine codebase. If you try to get there mainly by studying other people's code and one day sitting down and building the bulk of your final goal... you'll almost certainly fail.

Never heard of design paralysis but I guess it makes sense, given your description.

When I learned about Object Oriented Design for my first courses in Java, I sat down and read a book about Java for about 5 months. Then it finally clicked and all the theory I had read over time about how OOD works, suddenly made sense and I could pretty much utilize it all from the get got. It was the same concept I wanted to apply to learning Data Driven Design. When it clicks I learn incredibly fast and can apply the theory I've previously read with ease. I wanted to understand DDD in it's simple form. Which I do. I understand, in theory, how it works. I just can't apply it to code. It's just very different from OO.

I am not reading others code to get there but now I've reached a point where reading code is beneficial because I understand most of the theory behind it. It's just how I learn unfortunately.

My dream is to end up with a very flexible engine, yes. I am no fool though and know it will take time and effort to get there at least. It won't happen overnight ^_^

I just really hate the notion of starting to code something and have no one by my side to consult with over it. Which is why I like getting taught by teachers rather than reading books or courses on the internet. If I make things wrong from the get-go and learn nothing I feel I've wasted my time.

CEO of Dynamic Realities

When I learned about Object Oriented Design for my first courses in Java, I sat down and read a book about Java for about 5 months. Then it finally clicked and all the theory I had read over time about how OOD works, suddenly made sense and I could pretty much utilize it all from the get got. It was the same concept I wanted to apply to learning Data Driven Design.

The thing is that despite a similarity in names, object-oriented design and data-driven design are different classes of concept entirely. The former is more structured an idea than the latter. I'd also challenge your assertions regarding your understanding of OO at that point in time, because many academic programs focus entirely on the wrong parts of OO and without using it extensively on real software, you don't really build up a true, "battle tested" understanding. One can acquire a passable understanding of a concept through books and other references, but until you really start using things in practice you have no idea how much you don't know you don't know.

I just really hate the notion of starting to code something and have no one by my side to consult with over it. Which is why I like getting taught by teachers rather than reading books or courses on the internet. If I make things wrong from the get-go and learn nothing I feel I've wasted my time.

You really need to disabuse yourself of this notion. Making mistakes is a critical part of learning and if you try to avoid it, you're only holding yourself back. Build games, and when you have questions or get stuck, you can ask people on the internet. Stop being afraid of doing it wrong. Everybody has, everybody will continue to do it wrong and make mistakes -- professional engineers who have been building games for ten years still make them happily.

You are holding yourself back by not trying to train yourself out of that pattern.

You really need to disabuse yourself of this notion. Making mistakes is a critical part of learning and if you try to avoid it, you're only holding yourself back. Build games, and when you have questions or get stuck, you can ask people on the internet. Stop being afraid of doing it wrong. Everybody has, everybody will continue to do it wrong and make mistakes -- professional engineers who have been building games for ten years still make them happily.

You are holding yourself back by not trying to train yourself out of that pattern.

I think you misunderstand me on this point. I am not afraid of making mistakes.

I just really dislike making mistakes and learn nothing.

CEO of Dynamic Realities

A decent tool for XML serialisation is tinyXML2. There is no benefit in chosing JSON over XML in C++ as neither of them will deserialise directly to C++ data structures like JSON does in Javascript.

Data Driven Design can be used together with an OO hierarchy, by using a praser and a factory you can construct the instances the data is specifying and initialise them with data from the data.

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

A decent tool for XML serialisation is tinyXML2. There is no benefit in chosing JSON over XML in C++ as neither of them will deserialise directly to C++ data structures like JSON does in Javascript.

Data Driven Design can be used together with an OO hierarchy, by using a praser and a factory you can construct the instances the data is specifying and initialise them with data from the data.

That is a nice suggestion. But I was hoping I could carry it out in C# as C++ is still too overwhelming for me to comprehend.

Are their other similar libraries for C#?

CEO of Dynamic Realities

C#'s base class library has support for XML deserialization (I prefer the XDocument API over the actual XML serialization APIs, personally). There are 3rd party libraries for JSON as well, if you'd prefer. Search around and find something you like.

C#'s base class library has support for XML deserialization (I prefer the XDocument API over the actual XML serialization APIs, personally). There are 3rd party libraries for JSON as well, if you'd prefer. Search around and find something you like.

Yeah I knew about XDocument, so was curious to see if there were others.

CEO of Dynamic Realities

I use serialization quite often, both at the binary level and xml level. You can even do simple text readers. If you want to move into databases you can do a MySQL database or an Access database that drives your application.

XDocument is good. If you really want to understand the guts... write a serializer in C#. Compared with overall game development, file serializers are fairly straight forward.

For more on my wargaming title check out my dev blog at http://baelsoubliette.wordpress.com/

This topic is closed to new replies.

Advertisement