Sign in to follow this  
horrificmonster

XML Parser for Asset Manager

Recommended Posts

Hello,

I'm working on setting up my XML data to keep track of and load levels for my game. I'm using XML so I don't have to recompile/write code every time I want to add a level or change something about a level. I'm pursuing 'data driven' programming here. This is all well and good, except I have a phobia of messing something up before I even try. I've got some pseudocode here of what I'm going to attempt and it seems horribly inefficient. Maybe someone can quell my fears.

 

First up, I have an XML Parser that will take the level XML data, and return a list of elements. The root element being the levels themselves. Each level element in the xml file will have a level number, and children assets that will themselves contain information for how they should be rendered and the behavior they will exhibit during the game. So far so good (I think!).

 

Now we get to where the data meets the code. Here is a little pseudocode for my XML Parser (Keep in mind I'll be writing this in Java for Android).

 

XMLParser

Public

Accept the xml file from the user and return a list of elements.

 

Private

Initialize and pass the input file to the XMLPullParser library.

Reach each tag, and do one of the following:

     Create a new element object if it's a new tag

     Add the attribute to the element object using a key/value pair, ie: addAttribute("width", "300") I'll be using a hashmap for this. Do this for each attribute found.

     Add the value within the XML tags to the element via setValue("tagvaluegoeshere")

     For each element found within the root element, add the child to the current element. Recursively process each child element.

 

 

Element

Public

Interface for adding and removing attributes via key/value pair

Interface for setting value member

Interface for adding children elements

 

Private

Contain 'value', string value that is found between the beginning and ending xml tags.

Contain a list of children elements.

Contain a HashMap of key/value pairs for attributes found within tags.

 

 

Two things:

1. Does this seem like a sound way to handle xml files and putting them into a data structure? I want it to be general so I can use it througout my code. Then I'll set up a level class that will take the level element for the current level and pass it to the assetmanager to load the assets.

 

2. I'm also worried about resources. Are all of these hashmaps in each element going to use up a lot of memory? I'm not sure how Hashmaps are implemented in Java, but I'm thinking there is probably an array under the hood that is allocating enough memory for the hash function to address the maximum value that it can come up with. If I use the hashmaps to keep track of assets I've loaded in the assetmanager, is it going to slow things down? What is big O for a hash function in Java?

 

 

Any help/advice would be much appreciated, thank you!

Edited by horrificmonster

Share this post


Link to post
Share on other sites

2. I'm also worried about resources. Are all of these hashmaps in each element going to use up a lot of memory? I'm not sure how Hashmaps are implemented in Java, but I'm thinking there is probably an array under the hood that is allocating enough memory for the hash function to address the maximum value that it can come up with. If I use the hashmaps to keep track of assets I've loaded in the assetmanager, is it going to slow things down? What is big O for a hash function in Java?


If you're concerned with resources and speed, you have a bigger problem than hashmaps: XML itself. Manipulating strings, converting strings to integers, allocating new substrings each time you read a string, etc. XML has plenty of overhead that you can completely avoid if you use a different format.

Using XML offline and then converting to a more optimal format (such as protobuf) before shipping the game is better.

At runtime, you want to store your data in the most efficient data structures you can. If those are hashmaps, then fine. I would personally use typical class instances instead. Your top-level "get me an asset by name or numeric id" would be a map of some kind, though. Edited by Nypyren

Share this post


Link to post
Share on other sites

If you're concerned with resources and speed, you have a bigger problem than hashmaps: XML itself. Manipulating strings, converting strings to integers, allocating new substrings each time you read a string, etc. XML has plenty of overhead that you can completely avoid if you use a different format.

Using XML offline and then converting to a more optimal format (such as protobuf) before shipping the game is better.

 

Ah, so I should avoid XML.  I played around with serializing classes, but I was thinking it would be easier to tweak XML files during development. I'll look into 'protobuf' to see if that will meet my needs.

 

 

 


At runtime, you want to store your data in the most efficient data structures you can. If those are hashmaps, then fine. I would personally use typical class instances instead.

 

I have a question about this. If I'm writing my data structure to be as general as possible, is there a way for me to use class members in the way you mentioned? The XML Parser class doesn't know anything about what it's parsing, so it won't know what the elements, values, or attributes will be called. I couldn't say element.width, because the element may not even have a width member. I suppose my Level class could be privvy to where in each array the particular attributes are. So I'm assuming what you're saying is use it something like element.attribute[0]? I think I may drop the hashmap in favor of a resizable array, then the calling class will just have to know where in the array to find the data it needs instead of using a string to hash to an array location. I could even use enumeration so it's more like: element.attribute[WIDTH].

There may be a problem with this though, because if I use the array offset as a way to know what's what, then everything has to be in the right order. Every time. That may be more trouble that it's worth.

 

Thanks for the feedback it definitely got me thinking!

Edited by horrificmonster

Share this post


Link to post
Share on other sites


Ah, so I should avoid XML.  I played around with serializing classes, but I was thinking it would be easier to tweak XML files during development. I
It will.

 

Thats why usually big game engines have two pipelines. The easy XML/JSON/YAML one for development where you can easily either tweak directly or through tools, and the content pipeline that "compiles" all of those resources into fast and compact binary data, which is what the final product will be using.

 

If you want to go the markup route for developing, I'd recommend YAML (using SnakeYAML in Java particularly https://bitbucket.org/asomov/snakeyaml), its much easier to work with than XML.

 

HashMaps in Java are quite fast for big volumes of data (you can find benchmarks using them for storing millions of elements) but they're kinda bad on the memory efficiency department. They're implemented as an array of entries/nodes, each entry object holding a reference to the key, to the value, int hash and reference to next entry.

 

Thats 12 bytes for the object's header, 4 bytes for the key ref, 4 bytes for the value ref, 4 bytes for the int hash, 4 bytes for the next entry ref. 28 bytes, 32 bytes in total given 8 byte alignment (gets 4 bytes of padding). And for each entry you need a reference to it in the entry array, so thats another 4 bytes. And thats only for keeping track of the data, not even the data itself.

 

(On 64 bit systems it will be the same unless you're using big ass heaps (think 10Gb+) thanks to compact pointer representation on HotSpot at least)

 

But who cares man! Use whatever is easier to use right now. HashMaps, LinkedLists, whatever... (well not LinkedLists, that would be stupid). Later you can worry about memory usage if it ever becomes a problem. As long as you're aware of it, it'll be fine.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this