Jump to content
  • Advertisement
  • entries
  • comments
  • views

Using a Metadata Database

Sign in to follow this  


Hey everyone - long time no post. Sorry about that. :) I landed my dream job over a year ago and have been pretty busy since then. I'm working at Harebrained Schemes on the new turn-based Battletech game. The last one was over 20 years ago! :D Come check it out at:

Today I'm going to talk a little bit about a metadata database (That's really fun to say btw) and how you might use it in your projects.


What is a metadata database?

Let's start with a couple of definitions. Metadata is just data about other data. And a database is just a structured way of storing and accessing data. So... a metadata database is a structured way of storing and accessing data about other data... Is your mind blown yet?

Why use a metadata database?

I'm going to look at our units for this example. We have a bunch of units with lots of data stored in separate files. Our unit metadata contains things like the filename, the type of unit: mech, vehicle, turret - and tags that describe the kind of unit that it is: medium, sniper, jump_capable. Instead of hard coding our unit spawn points to a specific unit, we configure the spawn point to ask for a unit with the tags "mech" and "medium" to change things up a bit. Without a metadata database, we'd have to load up every single unitdef into memory, and then loop through all of the data to build a list of the units that match, then select a random entry. With a MDDB, we can write a query that returns the list of units that qualify without having to load every single file and load the unit that we need to spawn.

Also, once you put this type of data in a database, you can start writing sql queries to easily get at information. How many maps implement a particular encounter? How many contracts are written for the Escort contract type. Are there any events that can't be triggered because they rely on tags that are never awarded.

Some Library Options

We're using C# and Unity for our project, and these are the main tools we're using for our SQL needs.

Some TagSet Query Code

Here's some C# I wrote that builds a dynamic bit of sql based on the number of required tags and excluded tags. It wasn't trivial so I figured I'd share it. One requirement I'm not completely happy with is that the tags you ask for have to be in the database. At the start of the function I insert the ones that weren't present. It made the sql a little bit cleaner to look at and get right.  For our needs it's not a big deal because we'll only be asking for tags that we care about, but if an end user is typing in random stuff you'll want a slightly different approach.

Also I made one change to the Schema since after the graphic was made. I dropped the TagID and just use Name as the primary key. TagSetTag then drops its TagID column and gets a TagName column to point to Tag. I recommend that datamodel instead of the one shown, but our fans made the graphic for the data model and I didn't have a way of updating it easily. :) The code presented matches the old way.

public static List<TagSet_MDD> GetTagSetWithRequiredTags(this MetadataDatabase mdd, TagSetType tagSetType, TagSet requiredTags, TagSet excludedTags)
    // Get the tag rows for the specified tags so we can get their database ids.
    List<Tag_MDD> requiredTagRows = GetOrCreateTagsInTagSet(mdd, requiredTags);
    List<Tag_MDD> excludedTagRows = GetOrCreateTagsInTagSet(mdd, excludedTags);

    // Here's what the query will look like for 2 requried tags and 1 excluded tag
    from TagSet as ts
        -- Required Tags
        inner join TagSetTag as tst0
            on ts.TagSetID = rtst0.TagSetID
        inner join TagSetTag as tst1
            on ts.TagSetID = rtst1.TagSetID
        -- Excluded tags
        left join TagSetTag as etst0
            on ts.TagSetID = etst0.TagSetID
            and etst0.TagID='tag id 3'
        rtst0.TagID = 'tag id 1'
        and rtst1.TagID = 'tag id 2'
        and etst0.TagID is null

    string queryText = "SELECT ts.* FROM TagSet ts ";
    string joinText = string.Empty;
    string whereClause = string.Format("WHERE ts.TagSetTypeId = {0} ", (int)tagSetType);

    // Join to one instance of TagSetTag for each required tag.
    for (int i = 0; i < requiredTagRows.Count; ++i)
        joinText += string.Format(GTSWRT_RequiredInnerJoinFormat, i);
        whereClause += string.Format(GTSWRT_RequiredWhereClauseFormat, i, requiredTagRows[i].TagID);

    // Join to one instance of TagSetTag for each excluded tag
    for (int i = 0; i < excludedTagRows.Count; ++i)
        joinText += string.Format(GTSWRT_ExcludedLeftJoinFormat, i, excludedTagRows[i].TagID);
        whereClause += string.Format(GTSWRT_ExcludedWhereClauseFormat, i);

    // Assemble the query text and return the results.
    queryText = queryText + joinText + whereClause;

    List<TagSet_MDD> tagSetList = mdd.Query<TagSet_MDD>(queryText).ToList();

    return tagSetList;


Sign in to follow this  


Recommended Comments

There are no comments to display.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
  • Advertisement
  • Blog Entries

  • Similar Content

    • By RoKabium Games
      If you need to go back and re-read a message from Antalasia or Drengo that has been stated during game play, there is a "Messages" tab in the GUI menu where you can find a log of then.
    • By Estigio97
      Hello everyone. I am a novice developer with some friends, and we are doing an MMORPG. I don't know if this is the category of my question. But I suppose if I want to solve it without having to modify my skinned clothes and make a lot of adapted versions of my skinned Clothes for each combination to not make the game too heavy. We have separated them by pieces: Helmet, Chest, Arms / Shoulders, Gloves, Legs and Boots. As in all games of this type, clothing can be combined in different ways. The problem we encounter is that, for example, when wearing pants that are not close to the skin with a higher boot, it may happen that the model of the pants intersects with that of the boot, and that it seems that they cross each other . The question is: Does anyone know any plug-in or video or script that solves in some way this problem of creating clothing sets avoiding these types of conflicts that may arise? The only thing that occurs to me is that the boot "cuts" the area of the pants it occupies, that is to say that it correctly conceals the part of the pants. Or any other method.
    • By basnijhof01
      Okay so, I have just finished a Unity 2D course and I have this idea for a game I wanted to make.
      I want to make a Pokémon-like game that's called Pokémon Forest, it's called Pokémon Forest cause you will be able to play all the different Pokémon professors when they were young and trainers themselves. It's a huge open world game with every single region in it. Here are the main aspects of the game:
      Playable characters:
      Samuel Oak, who will be played in the Kanto region. His specialty will be the study of the relationships between people and Pokémon.
      Jonas Elm, who will be played in the Johto region. His specialty will be the study of breeding patterns and interactions between Pokémon.
      James Birch, will be played in the Hoenn region. His specialty will be the study of Pokémon habitats in correlation with human existences.
      Oliver Rowan, who will be played in the Sinnoh region. His specialty will be the study of Pokémon evolutionary patterns.
      Aurea Juniper, who will be played in the Unova region. Her specialty will be the study of Pokémon origins in relation to mythology.
      Augustine Sycamore, who will be played in the Kalos region. His specialty will be the study of the new type of evolution: "Mega Evolution".
      Noah Kukui, who will be played in the Alola region. His specialty will be the study of Pokémon moves and will sometimes get himself hit to study the moves.
      Emily Magnolia, who will be played in the Galar region. Her specialty will be the study of the new type of Phenomenon: "Dynamax".
      Base Story for every character:
      The choice between three starter Pokémon that have Grass, Fire, and Water as their primary types (they will encounter them in some way)
      A villainous team serving as the main antagonists of the game, so 8 different teams, 1 per character.
      A rival who starts their journey at the same time as the player, and is found multiple times throughout the game, so 8 rivals, 1 per character.
      The player challenges eight Gym Leaders, as well as the Elite Four and Pokémon Champion. Except for Noah Kukui, in Alola Gym Leaders are replaced by the island challenge.
      Other aspects of the game:
      It will be a 2D pixelated-like game.
      GTA 5 style, you have 8 different playable characters which will all be playable and you will be able to switch between them whenever you want.
      You play all the different pokemon professors when they were young.
      The professors were 11/12 years old, so the game will take place around 50 years ago.
      It's one big open world game where you can visit all the different regions: Kanto, Johto, Hoenn, Sinnoh, Unova, Kalos, Alola and Galar.
      Every professor has to defeat its own evil team and later on they have to work together to beat one BIG team: Team Rocket.
      The leader of team Rocket will be Giuseppe, Giovanni's father.
      Team Rocket will be unlocked after every character has defeated the elite four (they can only defeat the elite four after they defeated their region's evil team).
      You will be able to catch all 59 legendary Pokémon.
      All the gym leaders will be different from the original games(except the ones who were pretty old in the original games)
      The player is able to have six Pokémon with him at most, while additional Pokémon can be kept in the Pokémon Storage System, 8 different Storage Systems for 8 different players.
      The playable characters will be able to trade Pokémon and items with each other.
      Evolution via leveling up, trade, and evolutionary stones.
      Every region will have its original Pokémon, so for example Kanto will have the original 151 Pokémon.
      Every pokemon will have 1 or 2 types, there are 18 different types in total.
      There will be 742 different moves, all from the original Pokémon games.
      All moves will be TMs, except the Z-powered moves and moves who belong to that Pokémon (like Spirit Shackle for Decidueye).
      The HMs will also be replaced by TMs, the Pokémon who were able to learn the HMs don't need the HMs to do the specific action. For example Diglett could learn Cut and Rock Smash,
      so now it can Cut through trees and Smash Rocks without having to learn the move, you just need to have him in your party.
      There will be a friendship system, however, there will be no Poké Pelago and Pokémon-Amie.
      So this is it. I still need to make all the different storylines for the different characters. Does anyone have any tips on how to make the game not boring, cause it is a long ass game. I will give the different characters different specialties, so they will have different abilities or something. That will make it so that the player will want to play the different characters.
      Thanks in advance!
    • By Shaarigan
      Hello Forum,
      I reached a point in my rework of our build tool where I have to improve parsing different coding languages like C# and C++ to detect dependencies between files and projects. Maybe I will also add some kind of linting to the build tool. Unfortunately I'm fixed to an older version of C# to stay backwards compatible to older Visual Studio versions. This is the reason for not having the chance to use the Roslyn parser framework out of the box and I had to write my own "analyzer" to solve the task.
      That I'm now ready for refactoring, I want to make it into a real parser instead of Regex driven analyzer. For this task, I also had a look at parser utility libraries like Sprache and Pidgin to have some more convinience in declaring tokens and rules. I have written parsers on my own by hand many times before and they were always optimized to the special token/ rule and usecase. For example instead of just reading the next string and then comparing it to a keyword, then conditionally reset the stream or pass the token, I wrote something like this
      char[] buffer = new buffer[<max keyword length>]; bool MyKeywordRule(Stream data, TokenStream result) { if(data.Peek() != myKeywordFirstChar) return false; long cursor = data.Position; int count = data.Read(buffer, 0, Math.Min(myKeywordLength, data.Length - cursor)); if(count == myKeywordLength && buffer.Compare("keyword", myKeywordLength)) { result.Add(myToken); return true; } else { data.Position = cursor; return false; } } My hand written function is optimized in performance to first test the character, then fill a buffer with data that is as long as the keyword and finally test this for equality with the keyword. If I write this in Sprache or Pidgen, it will just read the stream, compare the result for the keyword and reset on failure, no early out will be performed or whatever speed optimization. If I want to test against a range of keywords or character sequences like for operators, in a generic solution it will read every statement of the sequence from the stream and test it while my hand written solution can fill the buffer once and test the result in a switch statement and early outs for the character count read.
      My question is if there exists a solution that can perform both, define rules in a convinient way like Sprache or Pidgin and on the other hand compete somehow with a hand written solution (it must not be as fast but in any way faster than simply replay the parts of a rule). I thought about a solution using C# Expressions where an operation for example my keyword, can be written as a sequence of pre-condition (testing the character), bootstrapping (reading the buffer for certain size) and validation (test if the buffer matches the keyword) and be merged together for example, so that a rule "A or B" dosen't test for a first then resets the stream and tests for B second but merges the pre-condition into
      IF not('A') and not('B') FAIL fills the buffer only once and then tests for equality in a switch-case statement. Or is it the best solution to keep writing such code by hand for each individual rule?
      Thank you in advance for your suggestions experts!
  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!