Jump to content

  • Log In with Google      Sign In   
  • Create Account

Asynchronous loading?


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
18 replies to this topic

#1 Telios   Members   -  Reputation: 398

Like
0Likes
Like

Posted 13 August 2012 - 11:25 AM

Hiya,


I'm currently loading entire levels synchronously, which is causing the game to freeze while everything is loaded and set up. Ideally I need to show a small animation or message while the level is loading and keep the game responsive.

I'm already have asynchronous file I/O implemented at the lowest level, but I'm not currently making proper use of it - my resource loader classes just request an asynchronous read and then immediately wait for completion.

It seems this asynchronous file I/O isn't the solution though - one level worth of data is roughly 20 Mb, which is read from disk very quickly. The code that's eating up cycles appears to be parsing XML and texture data for the objects and materials.

I was wondering if anyone can suggest a way of solving this? I might need a separate thread to parse the data, but this is likely to get tangled. If there's a simpler mechanism I'd prefer to give that a try first.

Also, perhaps importantly, I don't need to stream anything while the game is running - this is purely a load-time problem. Can anyone give their opinion?

Many thanks for any assistance!

Cheers.

[Edit]
In case it is helpful, this is what my existing resource system looks like. There might be a way to modify it to load asynchronously. Any class can be used as a resource - plugging it in just involves specialising the ResourceLoader<T> template for that class.

Posted Image

Edited by Telios, 13 August 2012 - 11:52 AM.


Sponsor:

#2 Telios   Members   -  Reputation: 398

Like
0Likes
Like

Posted 13 August 2012 - 12:16 PM

Thinking more about this, I might need to use a separate thread.

Instead of returning a pointer to the requested resource, the ResourceCache<T> could return a simple handle structure:

template<T>
struct ResourceHandle
{
  T* Resource;
  bool IsLoaded;
};

If the resource isn't already cached, a resource handle flagged as not loaded can be returned, with a null pointer. This frees the cache from having to always return a complete resource.

The cache could then submit a job to a separate thread using a producer/consumer queue, along with a callback. The thread can parse the XML or texture and pass a struct to the callback, which puts it on a 'ready to load' queue in the cache. The last part would be the cache being given an update() method, where it can create the resource on the main thread, using whatever API calls it needs.

If you wanted to load a resource and wait for it to become ready, it could look something like this:
ResourceCache<Material> cache(...);

// Later on...
ResourceHandle<Material> handle = cache.get("material.xml");
while(!handle.IsLoaded)
{
   // Keep polling for the material to be parsed and
   // given back to the cache.
   cache.update();
}

ResourceCache<T>::update()
{
   // Lock the ready-to-load queue.

   // Use whatever API is necessary to create GPU resources, etc.

   // Unlock.
}

Well, that got complicated pretty fast. Any comments are still very welcome :)

#3 ajm113   Members   -  Reputation: 308

Like
2Likes
Like

Posted 13 August 2012 - 01:43 PM

Example Method:

GameLoop:

if PlayerEnters New World Event
Unload Unseen/Not Using Data In Arrays (Small stuff like unseen meshes/untriggered entities/etc)
Create New Thread (Load XML/Textures)
Create New Thread (Load 3D Models/Sounds)

If Threads Done And Player is distance away from old world X.
Unload old world data completely.

If Player walks back to old world.
Safe Stop Threads If Running.
Unload the new world
Create New Thread (Load XML/Textures)
Create New Thread (Load 3D Models/Sounds)


Draw World

End of GameLoop

If you only add data to your arrays/pointers/vectors after you remove the stuff not in use, what have you. You drastically lose the chances of a crash of calling a outbound element and can still to continue to call models/entities in your game loop if the player decides to walk back even and takes less time to kind of reload the last level. I would create a second variable for the arrays something like size_t safeSize for determine what entities or models are safe to call. Then just add or subtract it when you add remove entities completely in a second thread. Debugging threads are a pain and I would write you a script if I could, but the chances are of it working are 50/50 if your using Win32. The best thing to do when using threads is simply experiment experiment experiment and debug till you know you got something completely uncrashable. Setting thread as a high priority fyi usually when adding a resource is helpful since it doesn't mess things up to other functions making a call to your pointers/vectors from what I experienced using threads.

Edited by ajm113, 13 August 2012 - 01:53 PM.

Check out my open source code projects/libraries! My Homepage You may learn something.

#4 dimitri.adamou   Members   -  Reputation: 329

Like
2Likes
Like

Posted 13 August 2012 - 03:42 PM

Wouldn't you be causing alot of unecessary locking? Instead of having it asynchronous, you can have it 'open'.

So when you go to your load screen, think of it like this

   Game Loop
	  While more data to load (Read from Queue or something like that)
		 Load Data for 50ms
		 Render Loading Page Progress

Something along that line

Edited by dimitri.adamou, 13 August 2012 - 03:43 PM.


#5 Telios   Members   -  Reputation: 398

Like
0Likes
Like

Posted 13 August 2012 - 06:43 PM

Thanks for the replies!

Example Method...


Interesting stuff, but I'm not sure I need that level of functionality/complexity right now. My levels are completely discreet - one gets loaded, unloaded, on to the next level. It's really just a load-time problem currently. Thanks for the tip about the priorities!

So when you go to your load screen, think of it like this

   Game Loop
	  While more data to load (Read from Queue or something like that)
		 Load Data for 50ms
		 Render Loading Page Progress


Thanks - I think that could work well. Instead of loading synchronously all in one go, I could trickle through a small number of resources each frame until everything is good to go. I need to keep a screen element animated while loading and I'm not sure how much I can do in the main thread, so I'll measure it and see for sure. Nice idea. I'm not too worried about locking, but threading always seems difficult to debug and profile for me.

Cheers!

#6 Hodgman   Moderators   -  Reputation: 30353

Like
6Likes
Like

Posted 13 August 2012 - 07:58 PM

It seems this asynchronous file I/O isn't the solution though - one level worth of data is roughly 20 Mb, which is read from disk very quickly. The code that's eating up cycles appears to be parsing XML and texture data for the objects and materials.

Ignoring threading for the moment, if you want to optimize load-times, you've got to eliminate as much on-load/parsing logic as possible. It's definitely possible to *not* parse textures or XML files on-load - you can move this logic into a data-compiler tool, which you run once at compile-time, instead of once every run.
E.g. you're parsing textual XML into some kind of binary structure in RAM, you can do this at build time, and then save that binary structure to disk. At runtime of your game, you can then load the binary file directly, with no parsing required. This should make loading your level near instantaneous.

#7 krippy2k8   Members   -  Reputation: 646

Like
0Likes
Like

Posted 13 August 2012 - 08:23 PM

Ignoring threading for the moment, if you want to optimize load-times, you've got to eliminate as much on-load/parsing logic as possible. It's definitely possible to *not* parse textures or XML files on-load - you can move this logic into a data-compiler tool, which you run once at compile-time, instead of once every run.
E.g. you're parsing textual XML into some kind of binary structure in RAM, you can do this at build time, and then save that binary structure to disk. At runtime of your game, you can then load the binary file directly, with no parsing required. This should make loading your level near instantaneous.


You can also have your engine do this at runtime and cache the results. In this case the first load will be slow, and any subsequent loads will be much faster. Of course you can ship your game with the binary versions so the user doesn't have to wait a long time on first load, but also make it easier to do modifications with the human-readable representation without having to run a separate compiler tool.

#8 Bacterius   Crossbones+   -  Reputation: 8836

Like
0Likes
Like

Posted 13 August 2012 - 08:47 PM

You can also have your engine do this at runtime and cache the results. In this case the first load will be slow, and any subsequent loads will be much faster. Of course you can ship your game with the binary versions so the user doesn't have to wait a long time on first load, but also make it easier to do modifications with the human-readable representation without having to run a separate compiler tool.

You could also make it so the engine keeps a fingerprint of the "readable" version that is currently compiled, and check against it upon initialization: if the hash has changed, go through the compilation process (hashing should be instantaneous unless your readable files are ludicrously large). Bonus points if you include both the readable and compiled version in the hash calculation, to ensure consistency.

This way people can mod the readable version easily and the engine will pick up on it, and they can also back up old compiled versions + the corresponding fingerprints for when they wish to revert and the engine won't try to recompile things that have already been.

The slowsort algorithm is a perfect illustration of the multiply and surrender paradigm, which is perhaps the single most important paradigm in the development of reluctant algorithms. The basic multiply and surrender strategy consists in replacing the problem at hand by two or more subproblems, each slightly simpler than the original, and continue multiplying subproblems and subsubproblems recursively in this fashion as long as possible. At some point the subproblems will all become so simple that their solution can no longer be postponed, and we will have to surrender. Experience shows that, in most cases, by the time this point is reached the total work will be substantially higher than what could have been wasted by a more direct approach.

 

- Pessimal Algorithms and Simplexity Analysis


#9 krippy2k8   Members   -  Reputation: 646

Like
0Likes
Like

Posted 13 August 2012 - 10:01 PM

Personally I like to just use the file modification time stamp for local files on the platforms that support it for better performance. Then it is also feasible to do the checks continuously in the background at runtime for live updating. Just compare the time stamp of the binary file vs the readable files. Some platforms like Windows provide support for watchers that will let you know when files in a directory are modified as well.

I just use hashes for syncing over the network.

#10 Hodgman   Moderators   -  Reputation: 30353

Like
2Likes
Like

Posted 14 August 2012 - 05:48 AM

Keep in mind that that the engine/game runtimes only make up half of an engine; the tool-chain is the other half (and the third half is documentation, support, bindings, etc Posted Image ).
Yep, for example - if users had to manually run XML files through some command line app to generate game-usable files, then your runtimes are probably very efficient, but now your tool-chain is now not very friendly.
I've used about 6 different engines over the past 6 years, and they've all had some kind of automated data-compilation pipeline, but have implemented them in a few different ways:
1) The runtime generated cache. During development, when the run-times load a file, they first check in a 'cache' directory for a binary version of the file. If a valid cached version isn't found, the original is loaded and compiled, then saved in the cache directory. When shipping the final optimised (non dev) version of the game, the compilation code is #ifdef'ed out and only the cached files are archived. If you want to support modding, you can also ship the dev build.
2) The build system. Just like you do for your code, you create a type of "makefile" for your assets, often for a custom built build-system, but possibly also for an off-the-shelf one (e.g. XNA uses msbuild). Whenever you've updated some assets (e.g. saved/edited them, or updated SVN, etc) you run your build-script, which uses timestamps/hashes etc to only recompile the necessary files. The game doesn't contain any parsing/compilation code (just simple binary loading code); all the parsing/compilation code is pulled out into separate tools that are used by the build system.If you want to support modding, you can also ship the build system.
3) The persistent build system. As above, but the build system sits in your system tray all day and watches your source asset directory. Whenever you change a file, it automatically recompiles it in the background. If the game is running, then when it's complete, it sends the game a message telling it to reload the modified file.

Back to threading though -- if you do want to parse/compile raw assets at compile time, then it should be fairly easy to parallelize. The inputs are the raw file and the outputs are the binary file, besides that, it's an isolated process, which is great for threading. When the async load of the original file is done, you can send a message to a worker thread (possibly by pushing it into a thread-safe queue) containing the original file's buffer, and have the worker-thread send back a buffer containing the binary file when it's done. The main thread can poll for completion once a frame, while it renders the loading screen.

#11 Telios   Members   -  Reputation: 398

Like
0Likes
Like

Posted 14 August 2012 - 09:04 AM

Thanks, very useful information.

2) The build system. Just like you do for your code, you create a type of "makefile" for your assets, often for a custom built build-system, but possibly also for an off-the-shelf one (e.g. XNA uses msbuild). Whenever you've updated some assets (e.g. saved/edited them, or updated SVN, etc) you run your build-script, which uses timestamps/hashes etc to only recompile the necessary files. The game doesn't contain any parsing/compilation code (just simple binary loading code); all the parsing/compilation code is pulled out into separate tools that are used by the build system.If you want to support modding, you can also ship the build system.


This seems very straightforward, and it will almost certainly make disk IO the loading bottleneck - which I can handle with overlapped reads.

3) The persistent build system. As above, but the build system sits in your system tray all day and watches your source asset directory. Whenever you change a file, it automatically recompiles it in the background. If the game is running, then when it's complete, it sends the game a message telling it to reload the modified file.


This seems very fancy Posted Image Is there a preferred way of the directory watcher communicating with the game? My first thoughts would be some kind of shared Win32 event, or a named pipe. Or perhaps the game listening locally on UDP?

Thanks again.

#12 Telios   Members   -  Reputation: 398

Like
0Likes
Like

Posted 14 August 2012 - 12:49 PM

OK - I've made a couple of changes to make my resource system work with asynchronous loading:

Posted Image
Instead of the cache returning T* it can now return a ResourceHandle<T>*, which tracks whether the resource is loaded or not. The cache owns a map of these handles.

When a resource is requested and the cache doesn't have it, it calls ResourceLoader<T>::preLoad(). This can issue the asynchronous read and return a 'not loaded' handle, which is stored by the cache in a separate pending list and returned to the client immediately.

When ResourceCache<T>::update() is called (once per frame), it checks the list to see which asynchronous reads have completed, and calls ResourceLoader<T>::postLoad() for those. That's where the GPU objects are created, for example. The resource is removed from the pending list, and it's flag is set to loaded.

So the overall flow will be something like this:
LoadLevel
   Request all resource from the cache, which returns the handles.
   While any resources are still being loaded:
	  Loop, draw the loading screen and call ResourceCache::update().

I think that should work. Seems reasonable?

Edited by Telios, 14 August 2012 - 12:50 PM.


#13 mightypigeon   Members   -  Reputation: 499

Like
0Likes
Like

Posted 15 August 2012 - 01:14 AM

Personally I like to just use the file modification time stamp for local files on the platforms that support it for better performance. Then it is also feasible to do the checks continuously in the background at runtime for live updating. Just compare the time stamp of the binary file vs the readable files. Some platforms like Windows provide support for watchers that will let you know when files in a directory are modified as well.

I just use hashes for syncing over the network.


I did this, and it was very fast and easy to implement, but artists got annoyed when trying to roll back older versions of assets and the cache wouldn't update itself (because the timestamp of the old file was older than the timestamp of the cached file). Hashing is definitely the way to go. (You can always do both and give the ability to turn off hashing to speed up loading)


#14 patrrr   Members   -  Reputation: 1004

Like
0Likes
Like

Posted 15 August 2012 - 07:02 AM

I did this, and it was very fast and easy to implement, but artists got annoyed when trying to roll back older versions of assets and the cache wouldn't update itself (because the timestamp of the old file was older than the timestamp of the cached file). Hashing is definitely the way to go. (You can always do both and give the ability to turn off hashing to speed up loading)


Probably stating the obvious, but you could just check the timestamps for equality. Posted Image

Edited by patrrr, 15 August 2012 - 07:03 AM.


#15 Hodgman   Moderators   -  Reputation: 30353

Like
1Likes
Like

Posted 15 August 2012 - 11:30 PM

Is there a preferred way of the directory watcher communicating with the game? My first thoughts would be some kind of shared Win32 event, or a named pipe. Or perhaps the game listening locally on UDP?

The implementations I've used have just used a regular TCP socket. One reason for this is because these engines have also been designed so that the game doesn't necessarily have to be running on your development PC (e.g. the game might be running on a console dev-kit). In development builds, these engines have implemented the entire file-system over TCP, so that you don't have to actually copy game data files over to the dev-kit's HDD constantly -- but this also sometimes comes in handy for the regular PC build when working in a team, e.g. if someone has a data error that they need help debugging, I can tell the game to connect to their data directory instead of to my own data directory.

So the overall flow will be something like this:
...
I think that should work. Seems reasonable?

Sure does Posted Image

#16 krippy2k8   Members   -  Reputation: 646

Like
0Likes
Like

Posted 16 August 2012 - 12:17 AM

I did this, and it was very fast and easy to implement, but artists got annoyed when trying to roll back older versions of assets and the cache wouldn't update itself (because the timestamp of the old file was older than the timestamp of the cached file). Hashing is definitely the way to go. (You can always do both and give the ability to turn off hashing to speed up loading)


I never check for a newer timestamp, just a different timestamp. What you mentioned is one reason, the other reason is that I've run into applications that like to change the timestamp manually and it's not always tuned to the system clock for some reason, so newer files can have an older timestamp.

You can then do a hash check if the timestamps are different to make sure the files are actually different before recompiling them.

Using hashes is probably fine if you're only doing it on load, though it will be a bit slower it shouldn't be too bad. But if you want to do live updating then it's a killer if you have a large resource set.

#17 Bacterius   Crossbones+   -  Reputation: 8836

Like
0Likes
Like

Posted 16 August 2012 - 04:18 AM

Using hashes is probably fine if you're only doing it on load, though it will be a bit slower it shouldn't be too bad. But if you want to do live updating then it's a killer if you have a large resource set.

Hashing isn't slow, really, if you implement it carefully. I've seen hashing schemes that left the actual data processing step in the dust, meaning they effectively have zero impact on running time. Of course, "carefully" is the trick word here, as always, so for an average implementation it'd probably slow it down a tiny bit.

The slowsort algorithm is a perfect illustration of the multiply and surrender paradigm, which is perhaps the single most important paradigm in the development of reluctant algorithms. The basic multiply and surrender strategy consists in replacing the problem at hand by two or more subproblems, each slightly simpler than the original, and continue multiplying subproblems and subsubproblems recursively in this fashion as long as possible. At some point the subproblems will all become so simple that their solution can no longer be postponed, and we will have to surrender. Experience shows that, in most cases, by the time this point is reached the total work will be substantially higher than what could have been wasted by a more direct approach.

 

- Pessimal Algorithms and Simplexity Analysis


#18 krippy2k8   Members   -  Reputation: 646

Like
0Likes
Like

Posted 16 August 2012 - 11:51 AM

I've seen hashing schemes that left the actual data processing step in the dust, meaning they effectively have zero impact on running time.


Most hashing is faster than data processing, we're talking about files that are pre-processed and thus don't require any runtime processing for the most part. (Except for possibly textures.) No matter how careful the implementation, hashing will have an effect on load times in this case. For live updating, the difference between checking a timestamp and reading and hashing a file that otherwise doesn't need to be read at all is obviously huge when being done continuously in the background alongside your game loop.

#19 Telios   Members   -  Reputation: 398

Like
0Likes
Like

Posted 16 August 2012 - 06:56 PM

Thanks everyone, this is great advice Posted Image

I've almost got the basic async loader working, but I've run into a problem.

Some of my resources have dependencies on others, such as a material being dependent on a shader program. In my original system I was handling these dependencies like so:

IResourceLocator* locator = new ArchiveResourceLocator("resources.zip");

ResourceLoader<Shader> shaderLoader = new ResourceLoader<Shader>(locator);
ResourceCache<Shader> shaderCache = new ResourceCache<Shader>(&shaderLoader);

// The material loader needs to be able to resolve shader references...
ResourceLoader<Material> materialLoader = new ResourceLoader<Material>(locator, &shaderCache);
ResourceCache<Material> materialCache = new ResourceCache<Material>(&materialLoader);

// A request for a material will now 'automagically' use the shader cache...


This is a problematic when the resources are loaded asynchronously, as a request for a material also has to wait for the shader dependency to be loaded.

When an asynchronous read has completed and the resource is ready to be 'loaded' for real, I will need to check whether it's dependencies are fully loaded before proceeding. I can see this "resources waiting for resources" turning into a big mess quite easily.

I don't think I have any other choice, but I was hoping to see what others think before I go for it.

Thanks again :)




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS