Jump to content
  • Advertisement
Sign in to follow this  
janta

Streaming resource loading / Multithreading

This topic is 4556 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello, I'm glad the forum is back to life :) Before I start designing sth, I'd like to request advices on the subject of streaming resource loading during a game. Here is the -common I think- design I've thought of (with pieces borrowed here and there as some people may notice) I plan to have a resource loader which runs on it own thread. That thread would be sleeping until one sends him a request to load some file. That request would have the form of stacking a resource identifier (most likely an integer) on the resource manager "loading command stack". It would go back to sleep when it's stack is empty, and also, it would know how to resolve to the real data file that the ID refers to. Since I dont want the caller to wait for the resource to be loaded (that could take a couple of seconds, even more), I thought that I could use an asynchronous pattern. What I plan to do is pass a pointer to an Interface along with the resource identifier. Somthing like:
void ResourceManager::LoadResource(UINT _uiResID, ResourceBase* _pCaller);

class ResourceBase
{
    virtual void SendMessage(int _msg);
}     
Calling this method would have the effect of waking up the loader thread and have it load the resource and fill appropriate fields of _pCaller in a thread safe fashion. When loading is complete, the thread would call:
_pCaller->SendResource(RESOURCE_READY);
to notify it that it can start using its resource. So what is your opinion about this first point ? Second point: I want to ensure that loading the resource is "load balanced". What I mean is that there is no point is having a separate thread loading a resource if that thread holds CPU monitoring (assuming I'm running on a single core) during all the loading process. So lets say I'd use the antique yet powerful fread() to read the data within some big file (like 20 MB) Is there any chance that a call like
fread(pData, 20 * 1024 * 1024, 1, pFile);
would yield the cpu's attention to another process ? If no, then it is pointless. And would that code be better (less efficient, but pure speed doesnt matter I just want it to be treated in harmony with the other threads:
for(int iChunk = 0 ; iChunk < iNbChunkInFile ; iChunk++)
{
    fread(&pData[iChunk*sizeOfChunk], sizeOfChunk, 1, pFile);
}
Or do you have any better suggestion ? As you can see I'm no expert of multithreading, yet I believe that it is the future and I want to learn :) Thanks for your reading ! Regards, Janta

Share this post


Link to post
Share on other sites
Advertisement
You can set your background resource thread to a lower priority than your main one. By default the OS will give each thread a time slice to execute in. You need to have the background thread yield to the main thread when its not doing any work.

The only time you'll be loading in a large a amount of data is during initialization. During the game, your resource streaming should be relatively small and shouldn't be too much of a problem. I think that the OS will suspend the background thread until the hard drive can fetch the data, but I could be wrong.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!