Archived

This topic is now archived and is closed to further replies.

Data files vs. text files

This topic is 4946 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Now, I''ve been working on a 2D RPG for awhile and one of the things I really wanted my engine to have the capability of is to be easily changed or modded. So at first I decided to leave all of my art and sound resources freely editable into respective sub directories within the main game directory. I would then have a editor/tool program that would allow the developer/modder to edit the datafiles to reflect the changes to the art (size, frames per animation, etc...). However, the other day as I was starting to add some functionality to my in-game menu, I realized... there is just too much work going on here to add a simple texture to this engine. I was still hard coding all of the resource variables and I about flipped on myself. So I have decided create resource managers, in that just basic linked lists (one for art, one for sound) that reads a list of files that need to be loaded for either sound or art and then loads them into the list with an ID number and then loads the textures or sounds as they''re needed or as directed by a loading a new area, or starting a new game, etc... So now it boils down to one last thing. My data files. I know that today, this is probably almost trivial, especially since it''s such a small project and really doesn''t have a whole lot of resources to load, but what is everyone''s view of using plain text files or (properties type file structures) for holding the data. Granted the raw data has to be parsed out of strings and does take a good chunk of CPU time as opposed to loading the raw data right out of the file. But my question is, in general, what is the pros and cons of doing something like this. Is there a point of too easy to modify... or too much load time activity... or even given the end user too much control over your game? I''d be intersted in hearing some opinions or views as well as pros or cons that I may have not seen. It''s definitely been an interesting project as well as a great learning one, and I hope to finish this engine soon so I can start working on some serious content for it.

I know only that which I know, but I do not know what I know.

Share this post


Link to post
Share on other sites
If the text files are plain numbers, characters, etc the don''t bother to use them. However...

...if the text files looks like:

character "GoodGuy"
icon icons/goodguy.pixmap
health 100
attribute main_character
[blahblah]

and

map Village
name "The Village"
width 18
height 30
for W use tiles/wood.pixmap
for . use tiles/grass.pixmap
for G use tiles/monster.pixmap
....W.....W...GG..
....WWWW WW...G...
[blahblah]


then do it (btw pixmaps are the X Windows bitmaps which are stored in plain text and they''re very easy to parse - also they''re supported by many drawing programs such as Paint Shop Pro and GIMP).

About speed, yes it will be slower. However a nice trick to overcome this is to cache the data in a binary file and each time you want to load the data, check the text file day. If it has changed (store somewhere the initial dates) then reparse it.

Share this post


Link to post
Share on other sites
Really, what you should do is work out a way to do things in binary and write an editor.

Text files are great, but they''re slow and can be edited by anyone. This not only compromises your game data, it can cause serious security issues if you''re not prepared to handle the situation where a user would REALLY mess up the files. Even then, you leave yourself open to holes you might not''ve thought of that clever users could exploit. Granted, you''re not safe with binary either, but at least users don''t have a clue what they''re editing then so taking advantage of security holes other than expecting the data to be fine all the time is less likely.

Furthermore, with more complex data you''ll probably find yourself being more productive if you have a utility to serve as an interface between yourself and the file. A map editor where you can draw using a tile palette, for instance, makes editing maps a real joke. I use VB to write up these kinds of utilities, since it''s fast and easy and performance isn''t an issue for a tool like that, personally.

Bottom line is that you''re better off with binary, but text is easier to edit. It''s up to you what you want to have.

Share this post


Link to post
Share on other sites
Exploiting the engine via text files sounds strange to me. However if it happens that means that features (bad or good) that the engine wasn''t supposed to support will be applied. Generally this sounds to me as a good idea for keeping up a community :-).

Share this post


Link to post
Share on other sites
If the game''s not networked, then there really aren''t any security issues (let them try to mess with the game as much as they want).

______________________________________________________________
The Phoenix shall arise from the ashes... ThunderHawk -- ¦þ
MySite
______________________________________________________________

Share this post


Link to post
Share on other sites
I use both text and binary files. The editor that I''m currently writing saves it''s level data as text which also includes editor settings and information that won''t be present in the final game data. Saving the files as text allows me to tweak them by hand if needed. Text files are also more friendly to a lot of the revision control systems such as CVS although this probably isn''t an issue if you are working alone.
The editor and/or associated tools generate binary files containing the raw level data which is faster to load and also has less risk of generating errors (such as syntax errors etc).

Share this post


Link to post
Share on other sites

If you do decide to go with text files, I highly recommend that you consider XML files. The "extensible" part of XML is a very nice feature for the programmer, because if you decide to start including more parameters in your files, your old data files are not necessarily obsolete.

Share this post


Link to post
Share on other sites
Don''t be afraid to use text files. If you are loading a small amount of information ( or even a lot ) and you are only doing it once on startup or level load, the cost is pretty negligable. It makes it easier to debug and test, and people who are changing your product can easily see how it all works. If you are writing a game where you are loading a large amount of data, the look into using binary data. I find it often makes sense to use text first, and if it''s taking too long, move to binary after the fact. As a side note, I''ve been using TinyXML to write and parse xml save files and such for applications and tools and I find it very handy.

Share this post


Link to post
Share on other sites
IMHO, the programming / conversion overhead of using binary files is too great, particularly during development.

Use text files until there is a reason not to, and then stop.

If that reason never happens, continue to do so in the finished product.

There is no security advantage in using binary files over text, as each can be edited by the determined attacker. To make it more difficult to modify them, store some kind of checksum (perhaps in a separate file), and check it at runtime. Crackers can still modify your game whatever you do.

Mark

Share this post


Link to post
Share on other sites
quote:
Original post by EvilSteve
If you do decide to go with text files, I highly recommend that you consider XML files. The "extensible" part of XML is a very nice feature for the programmer, because if you decide to start including more parameters in your files, your old data files are not necessarily obsolete.

That, and there are tons of wonderful XML parsers (many open source) already available for you to choose from. Xerces immediately comes to mind, since that's what I've used before at work, but it could be a bit heavy for your project. TinyXML is a lightweight library useful for general purposes. Or, if you're using .NET, System.Xml provides plenty of XML processing capabilities.

As you can tell, I really, really like XML

[edited by - Zipster on May 28, 2004 3:12:49 PM]

Share this post


Link to post
Share on other sites
Wow, I didn''t really expect to get that much of a response, but I sure did get some good opinions and views.

quote:

IMHO, the programming / conversion overhead of using binary files is too great, particularly during development.



I am finding this to be true for my project. The amount of work that is required for binary data files by having to constantly modify my editor applications and then modifying the files, and then modifying the data, and then re-''compiling'' the data with the editor. This is especially true as the file formats themselves are changing, and the time spent hunting down file structure errors is both time consuming and a pain. Whereas with a text file all I have to do is to create a new symbol and have the parser do whatever is needed for that symbol.

This isn''t to say that editors are bad by any means, I use editors for things like maps, to create them visually, however, trying to create visual editors for simple things such as character dimensions and stats, and just data for menus like coordinates of buttons etc... seem easier as a text format.

And I don''t think security is a big issue for this particular project because it''s intended to be freely distributed and edited or modded for creating one''s own adventures. Perhaps for the newbie game developers. I remember starting with things like that, just editing games that were already out there.

quote:

That, and there are tons of wonderful XML parsers (many open source) already available for you to choose from. Xerces immediately comes to mind, since that''s what I''ve used before at work, but it could be a bit heavy for your project. TinyXML is a lightweight library useful for general purposes. Or, if you''re using .NET, System.Xml provides plenty of XML processing capabilities.



I''m really not a fan of .NET or the .NET based coding and scripting languages and formats, so I don''t think I''ll be taking this road, but it does offer some interesting alternatives, perhaps for another project.

However, I do think I have reached a decision through these suggestions. I have decided on using a parser that is seperate from the engine and takes the text files and then translates them into a binary format that way. I''ll still need to give some more thought and experiment with different ideas.

Anyway, thanks everybody for the suggestions and comments.



I know only that which I know, but I do not know what I know.

Share this post


Link to post
Share on other sites
quote:
Original post by Daishi
I'm really not a fan of .NET or the .NET based coding and scripting languages and formats, so I don't think I'll be taking this road, but it does offer some interesting alternatives, perhaps for another project.



I very well could have misinterpreted this, but I just wanted to make sure you know that you don't need .NET in order to use an XML file for your game. .NET and XML are seperate technologies. One of the design goals of XML is that it is human readable, and easy to parse. As Zipster also pointed out, there are free XML parsers, and it's not too hard to write XML parsers yourself, as I have done. There's a simple tutorial about what XML is here, and of course you can read the full specs for XML here


[edited by - EvilSteve on May 28, 2004 5:21:26 PM]

Share this post


Link to post
Share on other sites