Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 20 Nov 2012
Offline Last Active Jul 24 2014 07:34 PM

Posts I've Made

In Topic: Idea for a demo of my 3D engine

17 October 2013 - 03:46 PM

Thank you very much for your replies! I guess there is no easy way round that I was hoping for, I'll just have to keep thinking and finding inspiration. :)

In Topic: Are there any standards to organizing game file and data chunks?

24 August 2013 - 11:49 AM

Yeah, there's nothing typical across all games studios ;)

I can tell you what I do in my engine tongue.png 
There is a lua file (or a collection of them), which list every asset which is available to the game. If some game code wants to load a model, it has to list the filename in one of these lua files first. If a file isn't listed, it isn't compiled.

assets = {}
AddModel( assets, "test" ) -- shorthand for: assets["test.mdl"] = hash("test.mdl")

The game wants to load a test model:

model = load(assets["test.mdl"]);
--this is the same as load(hash("test.mdl"))

i.e. filenames are not used by the file loading system. Filenames are hashed to produce a 32-bit integer name for each file.
When the game starts up, it loads the header of it's data archive file. This header contains a count (number of files in the archive), an array of filename-hashes, and then an array of file information (offset to each file in the archive, and it's size). These arrays are sorted by filename-hash, so they can be binary-searched.
When asked to load a file, the requested filename-hash is found in the array of filename-hashes (using binary search), and the corresponding size/offset is then fetched. These are then used to queue up an asynchronous read of that data in the archive.
The tools then of course need to be able to build this big data archive file wink.png
The content build system then runs though a tonne of rules to figure out what data to build, and how.
It starts with the lua tables above, e.g. it would see that test.mdl is required by the game.

It scans through it's "build rules" and finds one that says:
if you want to produce [foo].mdl, pass [foo].daegeo to a ModelBuilder.
Then it realizes that it needs the test.daegeo file, so it again scans it's rules and finds one that says:
If you want to produce [foo].daegeo or [foo].daemat, pass [foo].dae to a DaeParser.
Now it realizes that it needs test.dae, but there's no rule to produce it, so it assumes it must be a content file.
These rules look like:

local ModelBuilder = Builder("ModelBuilder")
Rule(ModelBuilder, "data/(.*).mdl", "temp/$1.daegeo")

local DaeParser = Builder("DaeParser")
Rule(DaeParser, {"temp/(.*).(daegeo|daemat)", "temp/$1.daegeo", "temp/$1.daemat"}, "$1.dae")

For "content files" like test.dae, they can be arranged in any way that the artists/designers feel like, with one rule -- no duplicate file names, because directories are ignored!


This may seem crazy, strange or stupid to some people... but we've found that using full paths for assets, and allow multiple assets with the same name is often a pitfall during production.
* during production, someone moves level1/textures/concrete.png over to common/textures/concrete.png because it's used in multiple levels.
-- if full paths are used, this action breaks level #1, and someone has to go in and replace the old path with the new path.
-- in our system, the file is just called concrete.png no matter where it is stored, so artists/designers can move files around and organize them however/whenever they want.
* during production, someone clones level1/textures/concrete.png over to level2/textures/concrete.png because they want to use it in their level.
-- if full paths are used, everything works, but we end up with two copies of the same file shipping on our final DVD!
-- in our system, the artists are presented with a warning/error, saying that level2/textures/concrete.png is being ignored due to a duplicate file name.
Anyway, getting back to the above model example, the build system now wants to find test.dae to use as an input.

When the build system starts up, it scans the entire content directory tree to build a map of file-names to full paths. It also subscribes to Windows notifications of any changes within the content directory, so it knows when new files are created, files are moved, deleted and modified.
Using this map, it discovers that test.dae is stored at content/test_level/test.dae. It loads this file and passes it to the DaeParser, which outputs temp/test.daemat and temp/test.daegeo. It then loads temp/test.daegeo and passes it to the model builder, which outputs data/test.mdl.
After building all the required files, it then loads up all the files inside the data directory, and writes them into the big archive mentioned before, which is used by the game.


If an artist modifies content/test_level/test.dae at any time, then the content build system receives a notification from windows, and can recompile the test.mdl file automatically.

Hi Hodgman, I'm currently writing my builder in a similar way using python, really helpful! Was just wondering what kind of intermediate format you use to export your scene information from a program like maya, for example exporting a scene graph or just the object positions?

In Topic: Shader "plugin" system?

17 August 2013 - 10:46 AM

In your ambient shader have something like:

#ifdef OPTION_04 // SSAO
vDiffuseMaterial.rgb *= 1.0-SSAO.Sample(InputSampler, i.vTex0).r;

Then you need to load a permutation of that shader with the macro "#define OPTION_04\n" prepended when your SSAO-plugin is enabled. In your config file you could specify which numbered options need to be enabled.

In Topic: Convert RGBA buffer to ARGB

05 August 2013 - 01:11 PM

Not looked at "LodePNG", but if it is uses libpng, libpng already provides all the transformations I have ever needed, like where to put the alpha bits (or add some fixed value if the source image had no alpha channel), so I would hope LodePNG lets you provide them.

LodePNG only consists of a .hpp and a .cpp file and doesn't use libpng.

In Topic: Calculating frustum corners from a projection matrix

21 June 2013 - 01:55 AM

Thank you! It was just that!