• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
PAndersson

Designing an asset loading system

10 posts in this topic

So I'm working on my first hobby game where I have to implement a proper asset loading system, in my previous projects I have had no problem fitting everything in memory at once. In this, I have estimated that VRAM will most likely be a limiting factor that makes that approach impossible. Though fitting it all in system RAM should not be an issue.

 

Now, I do not need every art asset loaded at once of course, and determining and limiting what needs to be loaded and what does not is not really what I wonder here. Instead I'm unsure on how to go about designing a proper asset loading system.

 

While I try to eliminate loading screens, beyond an initial, altogether by being smart about what is loaded and what is not the game still nicely needs to handle the sudden need for an unloaded art asset. Perhaps an object that requires it is suddenly spawned by an in-game event. The way I would like to handle this from the users perspective is to simply pause the game until it is loaded and pop up a "Loading" text on the screen.

 

I have two possible implementations in mind for this:

 

1. Finish executing the current frame, render the game state with the "Loading" text and do not do another update until the asset(s) are done loading asynchoniously. This does pose the limitation that any assets cannot be used in the frame they are requested as they may not have loaded yet, which can be an issue as the heaviest of them are both used for graphical and gameplay data.

 

2. Have the game-loop execute in a seperate thread, and pause that thread whenever an unloaded asset is used. The seperate rendering thread needs to handle this correctly (note that I already have a seperate rendering thread). This should pose few issues, but feels overly complicated for such a task.

 

So how do games usually do this?

 

Finally, I'm wondering if it is a good idea to have each State (ie startup menu, game running and the like) provide a seperate "loading state" that they enter when they want to show the user that they are loading something? It feels like it should not be their concern, but each state may need to show different loading screens and at least one of the different "loading screens" will need graphical information for the current game state.

 

0

Share this post


Link to post
Share on other sites

How I Would Do It:

 

Don't load everything into ram at once, bad idea.

 

Instead convert all assets to raw format and store in a compressed archive system for latter access.

Then using a virtual filesystem linked to your archive system, load each asset only when you need it.

 

Physfs is great for this, check it out.

 

I don't think giving you thousand of lines of code will help you, might only confuse you more. Instead you should impliment your own system, that way you learn what is going on.

 

In regards to your comment on state loading, that is what most commercial games do.

 

If you store all data in raw format you will not need another synchronise loading thread. The time it takes to load raw data is insignificant on modern systems, and one thread should do.

0

Share this post


Link to post
Share on other sites

I should add that a lot of the data is algorithimcally generated at runtime, in some cases the parameters used by the fairly time-consuming algorithm is not known until the last moment.

0

Share this post


Link to post
Share on other sites

Just remember that in a majority of cases anything created at run time that is rendered to a texture (via ogl or dx) will need to run on the main thread or you will get intermittent errors.

0

Share this post


Link to post
Share on other sites
Instead I'm unsure on how to go about designing a proper asset loading system.

 

...

 

The way I would like to handle this from the users perspective is to simply pause the game until it is loaded and pop up a "Loading" text on the screen.

 

 

I don't think it's 'proper' at all.  This means once in a while players will have to stop and wait for disk asset to load, looking at "Loading" on the screen all the time.  If this happens more than once during the game, I'd start cursing.

 

"Jump jump action! LOADING...FUCK! Action again! Shoot! Dodge! LOADING...#!@$!"

 

You might say, "but oh it's quick so user probably won't even see it".  Don't assume so.  File IO is one of the slowest IO operation.  If user's computer is set to sleep the hard-drive after 5 minutes, then it has to wait until the hard drive spins again.  What was a 10ms operation before becomes 5000ms.

 

What's wrong with loading screens?

0

Share this post


Link to post
Share on other sites
What's wrong with loading screens?

 

Wont really work well at all for the game I'm designing. As it is a space-strategy game with a fair bit of ground action, and as the gameplay is planned showing loading screens at any time except game startup would very much interupt the flow of the game. The planets themselves need a fair bit of detail due to the zoom allowed, enough that I cannot fit all of them inte video memory except on high-end vide cards. Fitting them all into system RAM is both trivial and required, as they affect gameplay as well as their apperance.

 

Having low-resolution versions of them, for display purposes, makes it possible to view them all at a high level of zoom. Selectily loading them into VRAM as the player zooms in on a region is fairly straighforward and any latency from that can be hidden. It is rather the sudden unanticipated need for new data (ie, some event suddenly creates a new planet somewhere and it's height and terrainmap needs to be generated with an algorithm) that would prompt the "Loading" text, and given that there will be very few instances where the game cannot predict what data it will need ahead of time the player will only very rarely see the "Loading" text at all.

 

The regular textures that represent plain images will be few in number and most fairly low in resolution.

0

Share this post


Link to post
Share on other sites

[quote name='PAndersson' timestamp='1357167726' post='5016884']
As it is a space-strategy game with a fair bit of ground action, and as the gameplay is planned showing loading screens at any time except game startup would very much interupt the flow of the game.
[/quote]

 

But this is exactly what you were trying to do. Having the word "Loading.." with the background of the game is not much different than a designated Loading screen.  It becomes the loading screens.

 

 

[quote]

It is rather the sudden unanticipated need for new data (ie, some event suddenly creates a new planet somewhere and it's height and terrainmap needs to be generated with an algorithm) that would prompt the "Loading" text, and given that there will be very few instances where the game cannot predict what data it will need ahead of time the player will only very rarely see the "Loading" text at all.

[/quote]

 

Depending on these unanticipated events, I'd try to hide the loading using some kind of animation.  For example, if a new planet is born, instead of showing "Loading...", show a specialized screen, like "Dam da dum dum! A new planet is born!! Explore it for new resources!" showing some quick animation that users can't cancel out -- during which the loading takes place.

0

Share this post


Link to post
Share on other sites
I have two possible implementations in mind for this:
 
1. Finish executing the current frame, render the game state with the "Loading" text and do not do another update until the asset(s) are done loading asynchoniously. This does pose the limitation that any assets cannot be used in the frame they are requested as they may not have loaded yet, which can be an issue as the heaviest of them are both used for graphical and gameplay data.
That is the traditional way of doing it. It's very good and safe. And by "safe" it means that memory allocation, deallocation, file I/O becomes centralized in one place (after/before rendering) rather than being sparsed all over the code.
If anything can be loaded anywhere, anytime, it's really hard to reproduce bugs, crashes, and you may probably be left with a few dangling pointers.
A good rule is to make everything valid during frame update (i.e. accessing an asset scheduled for unload won't hit a dangling pointer and you don't have to check for "null" ptrs every time) and manage creation destroys at the end/start of the frame.
 
2. Have the game-loop execute in a seperate thread, and pause that thread whenever an unloaded asset is used. The seperate rendering thread needs to handle this correctly (note that I already have a seperate rendering thread). This should pose few issues, but feels overly complicated for such a task.
That sounds awfully full of locks which means slow as hell.
If you want to do background loading (which is a good way to rule out loading screens), you weren't however that far from the truth:
As in the first method, you should request loading/unloads during frame time and tag that resources as "requested", then handle those requests outside the frame update. But this time, just do one lock and send those requests to the thread that handles loading/unloading.

When the bg thread finishes loading, the main thread will be informed when it finishes the frame and locks the mutext to send & receive data with that bg thread. The main thread will now tag the asset as "loaded" and can be used.

While the main thread tries to access a resource not yet been loaded, it should ignore it if possible (i.e. Graphics-related stuff, like a mesh not yet loaded) unless it's critical for the logic or physics (ie. terrain collision data). If that happens, then the main thread should stall until it is done (while checking for bg thread's response). You may want to load all data that could cause a stall at the beggining though.

Btw, this system could be extended (i.e. handling partially loaded data, like loading "LOD3" before "LOD0" and display that one until the highest quality is up; also handle "abort" to stop loading some resources) but I'll leave that for you to figgure.

Very important: Note that you shouldn't be requesting assets as soon as you need it. No background loading system can cope with that. You should request when you're reasonably sure you're gonna need them.
Sometimes this can be tied to gameplay (i.e. force the player to walk through a narrow corridor that leads to the next area, and he can't go back)
0

Share this post


Link to post
Share on other sites

I didn't read everyones reply, so sorry if this has already been said.

 

The way i've done this in the past is to have a game thread and a load boss thread and a number of load worker threads.

The game thread queues up assets ids that need to be loaded (newly visible stuff in an oct-tree), then passes that queue over to the load boss.

The boss thread knows how many worker threads are available, and gives each worker a thing to load. When the thing is loaded, the worker thread

signals the boss thread, who gives the worker the next thing in the queue. Once the entire queue is processed and all worker threads are asleep the loader thread lets the game thread know that it is safe to proceed.

 

How much loading time you have will greatly depend on how large the chunks you request are. Take a look at "Jack and Daxter", the whole world is constantly streaming in and out. Now, there are some edge cases where you can run into an area that causes another are to be visible and end up in a situation where the game needs to load; when that happens jack trips, falls and gets up. This animation is enough to load in the missing assets and is pretty damn funny. If that had been a "Please wait, loading" screen, the whole thing would have been a lot more frustrating.

0

Share this post


Link to post
Share on other sites

If loading the asset is going to take...

  • less than 10 ms, you can probably get away with loading it in the main game thread if you don't have to load more than one at a time.
  • more than 10ms, I'd make it a background process to prevent the game from stalling up. 
  • more than 1 second, I'd try to predict when the asset is going to be needed just ahead of time and pre-load it to reduce latency.
  • more than 5 seconds, you probably need to rethink how you're storing your data or use fast loading imposters until the data is available.

From my personal experience, unless I trigger a level transition, I get annoyed waiting for an unsolicited "Loading..." screen.

0

Share this post


Link to post
Share on other sites
I should add that a lot of the data is algorithimcally generated at runtime, in some cases the parameters used by the fairly time-consuming algorithm is not known until the last moment.

 

Could you give more detail about these algorithms and why you are unable to redesign you game to buffer the most common place data generated.  In the past I learned the hard way, not to do very many calculations during runtime.  Rather I had to rethink how I organized the game, and did so in such a way that I did all the calculations, then stored them on the hard drive in a buffer to be re-used. This sped everything up and was only slow during the first minute of play. If your calculations are not 1-1 or there are too many possible outcomes, then consider redesigning your game to limit these possibilites so you can implement a buffering system of some kind.

 

For example, say you have 100 possible states for a game object. In turn each state has dozens of sub states.  An algorithm for this object takes these states into account allong with it's locations and angle, then on the fly generates a new object to be drawn.

 

If you didn't notice, this is a terrible way to design a game, it is too slow.  In the real world we must take into account the limitations of everyday computers.  To make the example above work you need to limit the number of states from that example.  Then once you know the number of possible outcomes fit within 2^12 (A reasonable size), you can buffer.

Edited by kd7tck
0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0