Followers 0

# Unity Binary vs Text file formats

## 14 posts in this topic

0

##### Share on other sites
Well, text based formats are certainly way, way better for debugging, hand edits, etc. But at runtime or at least for production code, binary formats should be in use. So the ideal case is that your content build compiles the text formats into a binary format and packages that. Your game then loads the binary format. (Optionally your game can load the text format directly if necessary.)

Since most people (most hobbyists, anyway) don't bother with proper content build pipelines, they simply lack the step that converts to a binary representation and simply stick to text. That's my theory, anyway.

Binary formats rock at runtime.
0

##### Share on other sites
Quote:
 Original post by PromitWell, text based formats are certainly way, way better for debugging, hand edits, etc. But at runtime or at least for production code, binary formats should be in use. So the ideal case is that your content build compiles the text formats into a binary format and packages that. Your game then loads the binary format.

Agreed, all text formats should be able to be compiled into a binary version. I personally see no benefit to XML unless you need to share text data with outside sources. Elsewise a script (or even ini file) format can easily be made into a superior alternative.

Text for debugging, binary for release.
0

##### Share on other sites
Personally, I think the move to XML is a mistake, because XML is _way_ too verbose. If you want a text-based format, use something based on S-Expressions (used by Lisp). It's just as flexable, but it doesn't require repeating everything for an end tag, and if you make your scripting language also use S-Expressions, all your data can be stored as regular scripts (that possibly have access to different native functions).

Personally, I use binary formats for most things simply because they're easier to create with existing editors for images, models, levels, etc.

As far as community friendly, editing files manually is not at all community friendly, whatever information they store. You really would be better served by creating editors for things that don't already have editors.

For extensibility, binary formats can be just as extensible by using some kind of 'tagged format' somewhat like the TIFF image format or WAV audio format.
0

##### Share on other sites
Quote:
 Original post by PromitSo the ideal case is that your content build compiles the text formats into a binary format and packages that. Your game then loads the binary format. (Optionally your game can load the text format directly if necessary.)

A common improvement of that scheme is to load both text and binary formats (check for binary format first, then for text format), and to have the game output the binary equivalent of a text resource upon loading.

This has the advantage of allowing the game to output binary content highly compatible with its internal representation without code duplicates in an external compiler, and also allowing you to bundle the game with smaller, compressed text files that are turned into bulkier but faster-loading binary files when the game is first run (ideal when you want a small downloadable installer).

0

##### Share on other sites
Quote:
 - Accuracy: Floating-point numbers can be a real pain, but at least with a binary-based format you'll always get out the exact same value that was put in. With text-based formats, you'll probably have to limit the number of values in the mantissa so you don't have to read in 30 characters for a tiny value. Whilst the difference between 0.000000005 and 0.00000001 is pretty damned minimal, it can have an affect on objects with a paticularily small scale or where extreme accuracy is required (Very, very rare, but I have seen it happen).

please read what you wrote, it is totally wrong, you don't seem to understand how floating point numbers work! every number in memory can be reprisented with text description, but the 0.000000005 tolerance is NOT from conversion, it is what is actually stored in the data.

you just need to know when to use text files and when not to. if the file will be updated and tweaked often use text, if it is 3d model or texture use binary, it is simple as that. and try not to use xml, because it is overkill even for what it was originally intended.

0

##### Share on other sites
As long as you only use attributes in an element you dont need an end tag so you can make it quite small actually. I'm doing this for my own project file format and find it as compact as you can get it without removing the self descripting that follows it.
<Nodes><Node Type="Screen" Name="Screen1" Viewport="0 0 1 1" /><Node Type="RenderObject" Name="RenderObject1" Col="1 1 1 1" Pos="0 0 0" Rot="0 0 0" Scale="1 1 1" CamPos="0 0 -9" CamRot="0 0 0" CamFov="60" ScaleByAspect="false" /></Nodes>

It's still a lot more than a binary format of course. What I would like to see is a binary format of XML so you could use the exact same interface to you xml file just exchange the file. I think they are working on that. Not sure how it's going though.
0

##### Share on other sites
in most cases i think, it´s desirable to sacrifice some performance in order to debugg better

although its only reading and writing performance which is lost ... if you want you can make cachefiles out of your xml data, in order to load them faster the next time?

as for the numbers: saving in hex make it pretty fast to load into a programm (at least faster than with base 10)

and if HD-Speed is your bottleneck, try zlib
0

##### Share on other sites
What I don't like about XML is it's verbosity and that it has too many ways of doing the same thing (attributes are not really needed, they're just sugar, complicating the syntax). Most data that you need to store in a game is structured as a tree, which XML can represent, but in a very verbose way. A much simpler language for expressing tree structures can easily be thought up, such as the following:
nodes(    node(        type:Screen;        name:Screen1;        viewport(            from(x:0; y:0;)            to(x:1; y:1;)        )    ))
Ie. two types of node: one that can contain other nodes (the ones with paranthesis) and one that can contain a string (the ones with the colon/semicolon). The result is much less syntactic noise, less (one) special characters to 'escape', making it easier to read, write and parse. The only functionality that is lost is the ability to do text markup easily, such as specifying that a range of text should be bold (since strings cannot contain nodes descriping this).

ronnybrendel: Saving floating point values in hexadecimal kinda defeats the purpose of XML being easy to read. And besides I don't think the processing time required to parse a number means anything compared to the disk access.
0

##### Share on other sites
Quote:
 Original post by Joakim_arronnybrendel: Saving floating point values in hexadecimal kinda defeats the purpose of XML being easy to read. And besides I don't think the processing time required to parse a number means anything compared to the disk access.

your right. for integers it is a bit faster, and it requires less space in most cases

but you´re flexible with the size of your number 1-8 bytes for 32bit e.g. ... i mean it could eat less memory than binary (so you can say that loading a fully charged binary integer maybe as slow/fast as loading a hexvalue --- depending on the number itself) - and you can read it with your own eyes ;)
0

##### Share on other sites
Quote:
 Original post by XetickAs long as you only use attributes in an element you dont need an end tag so you can make it quite small actually. I'm doing this for my own project file format and find it as compact as you can get it without removing the self descripting that follows it.[...]
You entirely lose the ability to nest elements, though, which you don't do if you use s-expression format:
(Root  :Type "Dialog"  :SpeechFiles "/speech/bob/chapter2/"  (Dialog    :Name "Bob"    (Statement      :File "Hello"      :Text "Hello. How are you?"    )    (Choice      :FileSet "HelloResponse"      :TimeLimit 10      :Default :Neutral      '(        :Good "Good. How are you?"        :Neutral "Fine."        :Evil "Shut Up!"      )    )  ))
and if you're using Lisp, Code Is Data Is Code, so you only have to write one scripting system.
Quote:
 Original post by Delfi[...]please read what you wrote, it is totally wrong, you don't seem to understand how floating point numbers work! every number in memory can be reprisented with text description, but the 0.000000005 tolerance is NOT from conversion, it is what is actually stored in the data.[...]
Don't forget that converting text to binary involves a lot of math (for floating point numbers), and thus without a _LOT_ of effort, you will actually get less precision than is actually available in your floating point format.
0

##### Share on other sites
POV-Ray : Documentation : 1.4.4.7 Why are triangle meshes in ASCII format?

"...meshes use floating point numbers.
It might come as a bit of surprise that it is far from easy to represent them in binary format so that they can be read in every possible system...
...In order to store floating point numbers so that they can be read in any system, you have to store them in an universal format. ASCII is as good as any other."
0

##### Share on other sites
Quote:
 Original post by ExtrariusAs far as community friendly, editing files manually is not at all community friendly, whatever information they store. You really would be better served by creating editors for things that don't already have editors.

Well, what I should have pointed out is that it's a lazy form of community friendly. It doesn't really require any effort from programmers or marketing, whilst putting out a toolset post-release (Usually) requires some additional clean-up to be done by the programmers, as internal-only dev tools are generally raw, ugly and error-prone. Then it normally needs to have a look-over by the legal department to make sure nothing incriminating is released with the dev-tools (How many of you, even breifly for testing, have had and error dialog or log entry that contained profanity?).

It's more effort than most developers are willing to spend/publishers are willing to give, so most games either don't have toolsets released at all, or only have shitty little half-working plugins released. Text formats allow the community to take a stab at writing their own tools, which will probably get far more support than dev-released tools would.

(I'm not saying I agree with this practice, it's just the way it is a great deal of the time)

Quote:
 For extensibility, binary formats can be just as extensible by using some kind of 'tagged format' somewhat like the TIFF image format or WAV audio format.

For sure, as a matter of fact one of the primary goals of my (Binary) file format is to be extensible without breaking the format (My previous format was really getting on my nerves as I had to modify the exporter, synchronise the loader and re-export ALL the assets whenever I wanted to add a tiny new feature).
0

##### Share on other sites
Also, there is a big difference between 100% internal data that you do not intend users, modders, or 3rd party tools to read or manipulate and data that might be customizable, etc.

And in favor of XML, for data that is editable by external, tool or human, sources. ONE benifit is their readablitity, which can be thought of as a small amount of self-description .. although it is also a minus, since the words can be poorly chosen and confuse or decieve the user. The real benifit comes when you create a DTD / SCHEMA. In this way you can tell the world in a 100% agreed upon manner a very fundamental description of your datas organization and rules. Sure not everything that passes the schema is valid data (because schema hold almost no business rules), but everything which fails the schema is invalid data. So its a good starting poing. And your modding / tools can perform schema validation when loading or reading files, to let you know when you have broken anything.

Such checks can also be performed at game load, and in the case of my config files I embed a valid "default" file in the executable to be used in cases of invalid custom files. Oviously this is for small files you have few of, not large bodies of assests, which just have to be right or the program can't do whatever relies on them ... but that's the case no matter what you use.
0

##### Share on other sites
This question should not be answered with "always" or "never" type responses. There are cases where text-based formats (even XML) are extremely advantageous, and cases where binary is simply the only acceptable medium.

The equivalent-binary scheme that Promit mentioned is in very widespread use, and pretty much solves all the issues involved. An even simpler method is to store all the data in a purely binary format, and create an editor tool that expands the binary data to a textual equivalent representation. This allows trivial hand-editing of data, without introducing a potential point of failure in the content build pipeline, and without requiring duplicated code (i.e. game code has to be able to read the textual format). We're currently migrating to that approach.

There's a nice heuristic question that can be used to determine whether to take this route, or to use a format like XML: does an editor for this type of data already exist? I don't think this is by any means a hard-and-fast rule, but it'll handle the majority of the cases. (Note that I personally find sexprs to be more elegant than XML, but infinitely less practical, due to the lack of schema and editor support.)

Consider a real-world case: 3D models have plenty of existing editing tools. All of the decent modelling packages have plugin infrastructures that support custom file-format exporters. Therefore, using XML is not really useful here, since you can export from the editor directly to the needed binary format. (This gets subtly more complicated when including portability formats like Collada, but that's beyond the scope of discussion I think.)

On the flip side, consider a data format that defines animated cutscene/movie sequences to be rendered in-engine. There is no editor for such things, at least not one that is guaranteed to suit our requirements. This is where XML becomes an attractive option: XML parsing libraries are ubiquitous, so the cost of integration with the game code is going to be minimal; once the tree is loaded in-memory it can be trivially converted to any equivalent tree structure desired, meaning that there's no reason to worry about the XML format affecting the rest of your code architecture if you don't want it to; and a good schema combined with a good XML editor (like VS2005) gives you a validation tool and file format documentation all in the same place.

XML in this case truly helps eliminate duplication of information to a huge extent: since schemas can have embedded human-readable documentation, you don't need separate doc files; schemas instantly eliminate the need for sophisticated validation, and in the worst case additional runtime validation on the tree (post-loading) is trivial; and you've totally removed the need for a custom editor tool without precluding the introduction of one at a later date.

Use the right tool for the job; don't mangle the job so you can do it with one particular tool.
0

## Create an account

Register a new account

Followers 0

• ### Similar Content

• Do you like numbers?
Actually I do.
When I was child, I enjoyed to make multiples of 10 using license plate.
It was not a big deal, but quite interesting to me.
Decades later I planned to make a game and remembered that time.
So I decided to develop game based on that memory.
At the first time , it was just game to make 10, but I started to put various game like elements:
items, obstacles, store, trophies, leader board, etc...
And finally it becomes a game.
Some people tells me it's too difficult, but some people tells me it's enjoyable.
How does it looks like to you?
"Puzzle Ten Final"
Android Exclusive.

• By Kajamaz

Summary: EverEmber Reborn is a multiplayer online hardcore open-world action RPG.

Description:
The game takes place in the world of EverEmber, a fantasy world where you have no direction, only your skill and other players to either aid or hinder you on your journey. The game is a first person open world pvp game, with progression aspects taken from agar and slither. The world is not too huge, and because of that players will constantly fight and die, dropping their obtained gear and forcing to restart. The combat is skill based, with many places to explore. The best aspects from Runescape, Ultima, Mortal online, and the Elder Scrolls games are taken and incorporated. Everyone has their own adventure, it’s your choice how it starts and ends.

EverEmber Reborn is a sequel to the game EverEmber Online (check it out at http://www.everember.com/). They are very different in their concept and execution, but the fantasy elements and ideas are brought over from its predecessor. We have been developing EverEmber online for over 4 years, but it’s time we move on. We have also worked on various private server projects, so the developmental process is nothing new for us.

Team:
We currently have a fairly large development team already, 10 main developers, myself, Ozfer (server host, developer, IT tech), Juicyz (Lead Programmer), EMPHyperdrive (Programmer) Amit (Game Developer and Modeler), HugeJackedman (Game Developer), Gw1p (Game developer), Naitsirik (Modeler), Symbolizemusic (Musician), Aytimothy (Programmer). We also have a musician assisting us (Davide Severi) with our soundtrack and a concept and concept artist (Akram). SeeEnvy is a writer for us.

What we're looking for:

What we need are programmers as of this moment. We need someone who is determined on helping us program the game itsself. We have much of the networking done and we are using Photon. The language we're using is C#, so knowledge of C# is a MUST HAVE!

Requirements:

-Experience with Unity and its technology

-Interest in the games concept

-Ability to do one of the things listed in our needs, either C# unity programming, or networking assistance.

-Time to dedicate to the game, we don’t have a constraint but a few hours every couple of days will suffice.

-Be able to communicate in English, as all of our team speaks English either fluently or well.

-Have a method through which we could pay you eventually.

Eye Candy (In-game screenshots):

More Eye Candy/Concept Art:

If you are interested, please do contact us via Everemberonline@gmail.com or message me here on unity forums with what you can do for us, why we interest you, and whatever questions or comments you have. Feel free to join our website if you cannot help us in development, but instead can assist us by playing the game upon release or testing it!

http://www.everember.com/

See you in the land of EverEmber.

• Hi, we are small game develop studio called Drunken Monday. We are only two people and during the last year we developed a cross-platform multiplayer game: Slash Arena. And we're almost done. We are glad to present you our game:

Massively multiplayer online battles with swords and axes.
Simple arcade action! Dodge the attack and choose a perfect time to strike.

Battle Modes:
★ Deathmatch — Player vs All mode for 30 players. Score the highest damage and survive to win.
★ Arena 1vs1 — ranking duel for hardcore players. Your skills mean more than your high-level weapon.
Features:
★ Rapid battles. Play 5 minutes or 5 hours. It’s all up to you!
★ Swing your hammer and make 'em fly! Damage is calculated according to physical laws. Timing and distance matter!
★ Two types of attack — enough to make your enemy suffer from a painful combo! Master your skills.
★ Separate ratings for each Battle Mode. Monitor your progress.
★ Monthly rewards for the best players. Earn a pile of resources and unique character portraits.
★ Daily tasks. See if you can cope with them! >:]
★ Three characters with unique weapons and fighting styles. More characters are coming soon!
★ More than 30 upgrade levels for each character’s weapon and armor. Start with a simple leather jacket and get to the legendary royal armor!
★ Character’s appearance changes each 3 levels. Everyone will see how cool you are!

Game available on: Facebook, it passed greenlight and coming on Steam, soft-launched on GooglePlay and AppStore in Russia ( If you contact us we will send you .apk or testflight invitation ). Also take a look at Slash Arena: Online and Drunken Monday web sites.
• By NA-45
EDIT: We've found a designer/composer and an artist.  I'm looking for one more artist!

I'm currently working on Metroidvania style game that I was inspired to start by Hollow Knight and Beksiński's art.

It's built in Unity using C# and has quite a bit done already.  I'm handling the programming myself and have a working model (besides combat which is a WIP) that can be expanded greatly depending on where we decide to take the project.  You can see the current test area here: https://streamable.com/mp5o8  Since I'm not artistically gifted, its all rectangles but can easily be skinned once we've desired on designs.
I have professional experience using Unity and C# working on both a released game and a prototype as well as having extensive Java knowledge.  I also dabble in Python with a little bit of C++.
I have worked on and completed many projects before, the most recent being a 2D stick fighting game written ground up in Java Swing (don't ask why): https://www.youtube.com/watch?v=V4Bkoyp_f0o
I'm looking for a 2D artist (potentially more than one) to create concept and game art and a designer/writer who can help flesh out the story as well as map out and create challenging and eye catching areas.  I can handle most if not all of the programming side of things though if there is anyone who is extremely passionate about this sort of thing, I'd consider splitting the load.
The end goal is a completed game that can be sold however profit isn't really a concern to me as it's mostly a labor of love from my part.  Any profits would be split between team members however that's pretty far off so don't make that a reason to join.
______________________________
The story I have in mind is something like this:
A man wakes up in a chasm that stretches seemingly endlessly in both directions lined with enormous statues.

He discovers a temple with text above a closed gate that tells of the failed kingdom that lies below.  After finding a way around this, he drops down into the subterranean kingdom.  Adventuring through the labrynth below, he comes across different cities in which the residents succumbed to different sins such as Greed, Wrath, etc.  Each city tells a story of how its fixation on something lead to their demise leading up to a fight with the personification of their mistake.
______________________________
An very rough idea for Waterways, a potential area:
- To enter you must be wearing a pair of glasses that you find somewhere earlier in the ruins.  There are similar glasses found in every home.  Everything appears incredibly beautiful however something seems wrong.  After triggering some event, the glasses break and it's revealed that the glasses are made of some sort of stone that makes everything appear differently.  The city is in ruins and absolutely disgusting as everything was neglected.
- The only thing that remains intact is in the center of the city, an incredible statue of a goddess holding up a large sphere of the same material that was used for the glass.  You slowly learn the story behind the statue: the goddess came from the sea that the city lies on and brought prosperity to them.
- After opening up the the temple of the goddess that lies right on the edge of the waters, a giant sheet of the glass covers an opening in the back of the temple that reveals the goddess behind it.  You shatter the glass and it becomes apparent that the goddess is actually a disgusting creature half beached and mostly immobile that appears to secrete the material that makes up the glass. Fight ensues.
______________________________
The combat is pretty up in the air and part of the reason I need a designer to bounce ideas off of but I think it will be something like this:
- 4 orbs equipped at a time
- 2 orbs selected at a time
- Pressing the cast button will cast a spell determined by the 2 orbs that are selected
- Spells cost mana however you can use spells with 0 mana and it will cost health instead
- These spells in addition to being useful for combat, are the Metroidvania "gating" metchanic.  For instance, one of the conceptualized spells is a water orb + water orb to create a ice pillar that can be either used to block projectiles/enemy paths or to jump on to reach high areas
______________________________
If you're interested or have any questions, contact me through discord.  My id is NA-45#3692.
• By sZokka

Radio Rabbit is a local coop shoot ‘em up where two players control one more or less combined character. The character exists out of a rabbit’s body and a floating, still to the body connected, giant eye.
Each player controls one of them.

The rabbit’s goal is to fly safely through the level and to avoid enemies to reach the goal.  The Eye on the other hand can shoot. He is the one who clears the way. One character can move the other can shoot. So both player need to work together to fight of evil creatures and to complete the level.

•    explore the level to find the key which activates the portal gate
•    escape through the portal before the timer runs out
•    if you are to slow, the nuke will explode
•    use your character abilities, the rabbit can boost while the eye got the vision
•    you’ll get more powerful abilities from items such as a supershot
•    shoot as many enemies as possible to gain score
•    remaining time at the end of each level gets added to the score

Features
•    2 Player couch coop
•    4 level + tutorial
•    an epic boss fight
•    fully gamepad supported (XBox or equivalent)
•    local high score

Grab a friend and check it out!