Back to General and Gameplay Programming

Loading Rather Complex Objects Quickly

General and Gameplay Programming Programming

Started by Angelic Ice August 10, 2016 10:24 AM

7 comments, last by sirpalee 7 years, 8 months ago

Angelic Ice

929

Author

August 10, 2016 10:24 AM

Hello forum!

I'm looking my way around to load a level.

I did this quite some times already. However, this time I need a bit more of information to be loaded.

One time, I used XML and realised, it is too verbose for my undertaking.

Another time, I was going with a simple 0 1 0 2 0 0 0 0 ... per line for the physics. Each number describing how it shall behave.

Additionally providing a graphics file, telling what graphics sheet to use and which coordinates for which tile.

This time, it is a bit different.

I have objects, that have a function, when they are being hit by the player and a different one when hit by the mouse.

Therefore, every object in the level-file will create its own Lua object, this will subscribe to its needed processes in the C++ engine.

Now, how would I do this rather "clean"?

Every object needs the following information:

Position (x, y)
Event-Trigger (varying: on_entry, on_click, in_front, ...)
Animation-Sheet path
Graphics-path

Now, I feel like parting these pieces of information can become a bit messy.

Parting in terms of handling one file for every important piece.

On the other hand, having it all in one place... can become very verbose.

I'm planning to replace manual editing with a level editor anyway!

Though, I'm trying to think about an elegant way to implement the way the level is stored.

Any suggestions to provide a rather quick loading for this matter?

Thanks for reading my thread! : )

Kylotan

10,510

August 10, 2016 10:55 AM

JSON is a good alternative to XML - still human readable but less verbose.

I strongly recommend against a binary format at this stage. They can be much quicker to load but they will give you a lot of grief during development until your formats and tools are nailed down.

I would be inclined to store Lua scripts in separate files, and reference them by name from your level/model files.

Don't worry about the 'quickly' part. I doubt your objects are really as complex as you think they are.

BitMaster

8,652

August 10, 2016 11:09 AM

I recently stumbled over flatbuffers, although I have not yet tested them myself.

Bregma

9,461

August 10, 2016 11:38 AM

JSON is a good alternative to XML - still human readable but less verbose. I strongly recommend against a binary format at this stage. They can be much quicker to load but they will give you a lot of grief during development until your formats and tools are nailed down.

QFE

Step 1: make it work. Step 2 (advanced users only): make it work fast.

Get your game 'finished' then polish level loading performance. Until then, benefit from a widely-used well-tested well-supported human-readable format like JSON or YAML. If your loader is well designed, it's just a matter of substituting one well-tested concrete loader for another at the same interface later on when you no longer need to be manually tweaking levels or debugging your pipeline.

Stephen M. Webb
Professional Free Software Developer

Hodgman

52,717

August 10, 2016 12:21 PM

Therefore, every object in the level-file will create its own Lua object

Lua is actually a fantastic DDL as well as being a fantastic "scripting"/extension language. Often it's cleaner than JSON due to the ability to name a local table and reuse it within several other tables. e.g.

--cave_spawns.lua -- returns a table indicating that the game should spawn 5 ogres with swords, 1 with a bow, and two skeletons with bows:
local sword = { attack = 20, range = 1 }
local bow = { attack = 10, range = 20 }
return
{
  { count = 5, type = "Ogre", weapon = sword },
  { count = 1, type = "Ogre", weapon = bow },
  { count = 2, type = "Skeleton", weapon = bow },
}

Seeing you want to use Lua for your gameplay code, this will be super-fast, as instead of worrying about searialization or fancy binary formats, the Lua compiler is your serializer and implements your binary data format, and the Lua runtime is your deserializer :D

I'm planning to replace manual editing with a level editor anyway!

We actually have quite a few editors that output "lua code" like this, as a human readable data format a la JSON.
If you're doing that, just make sure to always start the file with something like:
--This file was automatically generated by <level editor>! EDIT AT YOUR OWN RISK!
Because if the level editor loads and saves it, it will save it the way that it wants to -- and "stylistic" changes made by hand will be lost, and anything that the editor didn't understand during loading will also be lost.

. 22 Racing Series .

SeanMiddleditch

17,596

August 10, 2016 11:59 PM

Get your game 'finished' then polish level loading performance. Until then, benefit from a widely-used well-tested well-supported human-readable format like JSON or YAML

I'm going to expand on this:

Generalize how you load data. The first version you support should be an easy to read and understand text format. XML, JSON, YAML, TOML, HJSON, whatever your heart desires.

Then add a binary deserializer, and write a tool that converts your text formats to binary. DO NOT GET RID OF YOUR TEXT FORMATS. In fact, don't even commit your binary data to source econtrol - they are converted resources, not source assets. Checkin the text assets and - to distrubte the game - process the text assets into binary ones using an asset pipeline tool. This tool is also what you'd use to compress textures, convert models to your engine's optimized format, etc.

Text is what you save and edit. Binary is what you ship.

You can't diff binary files, you can't merge binary files, you can't inspect binary files. They're an optimization. They're crap for editing and development.

Sean Middleditch – Game Systems Engineer – Join my team!

Kylotan

10,510

August 11, 2016 10:12 AM

Then add a binary deserializer, and write a tool that converts your text formats to binary.

http://www.7-zip.org/ (or similar)

And I'm only being semi-facetious. For many serialisation problems, the bottleneck is the time spent pulling it from disk. When you compress a JSON file you'll probably cut it down by 80%. And when you put a bunch of them together into one archive you save a lot on disk seek time.

You might still need a proper binary format later, but try compression as a cheap intermediate step first.

KaiserJohan

2,320

August 11, 2016 12:38 PM

Concerning serialization/deserialization of objects, If you are OK using Boost you could use Boost.Serialization (http://www.boost.org/doc/libs/1_36_0/libs/serialization/doc/index.html). Supports serialization of stl containers from the start and very easy to use.

sirpalee

761

August 12, 2016 10:17 AM

Encode your data to something like z85 when storing in your choice of file type, use blosc for fast compression / decompression. And so on.

shaken, not stirred

Loading Rather Complex Objects Quickly

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Loading Rather Complex Objects Quickly

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines