• Advertisement

Hierarchical Data in a Web Application

Recommended Posts

I am creating an HTML5 game. Part of the content of the game is based on some hierarchical data. I need to take this data and load it into a tree data structure. I have my tree data structure working but I need to figure out exactly how to store the data.

After much frustrating trial and error I realized that for all intents and purposes it's not possible to pull data into Javascript. So I couldn't have an XML file sitting in the same directory as the html and javascript files and load it and parse it. I found it utterly amazing that there was no workaround for this other than using node.js or something which I don't currently want to do.

So given that that is impossible, which again is amazing since an HTML file can load in javascript files and images and other various sources, how would I store and process my hierarchical data?

For the purposes of the question, assume the hierarchical data is basically of a form that would be conducive to the XML format.

 

Currently my solution, which is extremely ugly and cumbersome is to handcode the data into javascript like this:

var text =
"<note>"+
"<to>Tove</to>"+
"	<from>Jani</from>"+
"<heading>Reminder</heading>"+
"<body>Don't forget me this weekend!</body>"+
"</note>";

 

Edited by aptpatzer

Share this post


Link to post
Share on other sites
Advertisement

You certainly can read in data via Javascript - you just don't have full access to the underlying file system when you're running in a browser, and that's for security and privacy reasons.

The standard way of storing data in Javascript is the JSON format. This is actually just Javascript itself, except in a form that is very amenable to storing configuration data.

Since you can import Javascript objects easily enough, your data files can be .js files, or stored inline in the HTML, etc.

Share this post


Link to post
Share on other sites

So does that mean I should store it the way I did in my code example?

My only issue with that is it's cumbersome. It's because javascript doesn't allow new lines in strings without escape characters.

Share this post


Link to post
Share on other sites

Given that I said you should use JSON, and your code example is XML built in a string, then no. Did you follow the link in my message?

Handling new lines in strings is a common problem, and escaping the new lines is the most common approach. XML lets you have new lines in text nodes but often you end up manually removing these extra nodes since they're only there for formatting the source file, so it's a tradeoff.

Share this post


Link to post
Share on other sites

I am familiar with JSON strings. But I don't see how I can use them here.

Currently the data is in a huge text file where child/parent relationships are expressed via tab indention. It's an ugly mess.

How do I take that and convert it into a JSON. And why would JSON be any better than just prepending each line with tags and treating it as an XML string and using DOMParser().parseFromString(text,"text/xml")?

Share this post


Link to post
Share on other sites

JSON is a hierarchical data format, like XML, HTML, or many others. If you look at some examples it should be relatively clear how they operate, and you can load them directly into objects in Javascript (which is why the format is so popular). It's literally one parse call to take it from a string to a hierarchy of objects.

Share this post


Link to post
Share on other sites

I see. But is that better than using XML instead?

XML can also turn into an object with DOMParser().parseFromString(text,"text/xml")

I'm just trying to understand the advantage of using JSON here.

Taking my raw text data and converting it into JSON format seems like a lot more work than just appending and prepending tags to convert it into the XML format. But I'm sure I'm probably not understanding what you're suggesting here.

Share this post


Link to post
Share on other sites

How are you generating the XML at the moment? I sure hope you are generating it in an automated way. If not, I strongly suggest you do so because manually decorating data so it's valid XML will become old real fast.

You can easily modify your XML generation code/logic to generate JSON instead. Or even better, just output straight JavaScript that creates and populates an object immediately. No need for parsing later on.

As mentioned before, you can easily put that generated code/data in its own separate .js file and include that in your HTML page as you would with any other .js file.

Share this post


Link to post
Share on other sites

Yes just decorating it manually for the time being.

Kylotan didn't mention why he preferred JSON for this application to XML but I'm going to assume he had a good reason and switch over to JSON for what I'm doing.

As you suggested, I'd like to automate the conversion process. It took me a little bit to figure out a nice algorithm for converting hierarchical data whose hierarchical nature is expressed by indentation to a tree, but once I got that working, I realized I should be able to make something that generates a JSON.

My algorithm involves a stack of references to nodes plus one extra reference not in the stack. I could put that extra one on the stack and make the algorithm more expressible, but that just results in a lot of superfluous pushes and pops.

Share this post


Link to post
Share on other sites

There are generally 2 very good reasons to use JSON instead of XML when dealing with (client-side) web applications:

  1. JSON stands for JavaScript Object Notation and as the name implies, it's a direct relative of JavaScript and therefore very suitable and easy to deal with in JavaScript code.
  2. It's a lot less verbose than XML, while still human readable (all the opening and closing tags required for valid XML take up a lot of space while not necessarily offering much benefit) . Depending on how much data we're talking about, that can make quite the difference in bandwidth requirements for your application (number of bytes being transferred over the wire to make your application work).

Creating an 'output generator' that processes your data and outputs it to either XML, JSON or code (e.g. JavaScript) is definitely worth the effort if you expect you'll have to do that more than a couple of times in the future. It's also a nice exercise if you've never created a 'data processor' like that before.

Share this post


Link to post
Share on other sites

#2 would probably be the main draw for me

I currently have about 1.5 million unique users per day that perform client-side operations that result in modifications to their personal "tree" of data. So performance is definitely an issue. Average user modifies their information about 80 times a day and the data I'm gathering from them has enabled me to do some optimizations that preclude the need to worry about this issue in the short term. In about a month, the people offering to buy this service from me predict we'll need better methods since volume is roughly doubling per week.

Ask me a week ago and I'd be more stressed out about how I was going to solve this since it was just a personal project. I also wasn't sure how I was going to make rent :P

Now I can hire people and I have an unbelievably huge offer that I'm considering taking for acquisition of this service.

If I don't accept, it'll be because retiring is less important to me than remaining an active industry-shaping force in a field I am passionate about, and I'll be hiring several full time people to take this workload off my hands.

Share this post


Link to post
Share on other sites

Wow, that escalated quickly!  :o

Good stuff! :) I'm very glad to hear it made such an incredibly positive turn.

And yes, with that many users and data permutations you'll want to seriously rethink your data structures and representation. I wish you all the best and hope it'll continue growing steadily!

Share this post


Link to post
Share on other sites

In case anyone is interested (probably not) I ended up declining the offer, hiring some staff to expand production. Then while we were shopping for an office space, the original offerers came back with a 180% hike to their offer (almost tripled).

I caved. As fun as the project was and as much as I was loving having correspondence with big people in the industry that I've admired for years, it's time to provide for my family.. and extended family.. and friends.. for multiple lifetimes!

Edited by aptpatzer

Share this post


Link to post
Share on other sites

For anyone reading this. You can place json or xml files just fine into your resource folder next to your html and png files ON THE WEBSERVER. Then you load it using xhr. Set cache timeout to something sensible. User data goes to local data storage "COOKIES". This is basic stuff. There is no: xml vs json. XML has some serious schema definition stuff and xpath and namespaces. If you sure you do not need this or want to reinvent this in json, just go ahead.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Advertisement