Jump to content
  • Advertisement
Sign in to follow this  
tseval

threaded texture loading

This topic is 3714 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hey folks, I'm trying to do asynchronous loading of texture data and other resources from disk. The way I do this is to have a separate thread with an input queue where I put all the resource loading requests. When the data has been loaded in the data reader thread, the rendering thread can start using them. I guess this is a pretty common way of doing things. However, I'm experiencing big problems with frame rate stuttering. Even if the textures are loaded in a separate thread the rendering thread is suffering major fps loss when the data reader thread is busy. These problems are the same on a variety of test machines, many of them fast multicore computers. I have run this code on Linux, Windows and OSX with pretty much the same results. I have tried inserting sched_yield() calls and sleep/usleep() calls in the data reader thread, and then it runs smoother, but it seems to be heavily downprioritized by the kernel so that the data loader runs way too slow... Is there anything here I have missed, any tips or help would be highly appreciated. Cheers!

Share this post


Link to post
Share on other sites
Advertisement
Does your other thread need to access the HDD? If so, it will obviously be bottlenecked by the (sloooooow) access time to the disk.

Share this post


Link to post
Share on other sites
All HDD access is put in the data reader thread. There can be multiple reader threads that access the HDD, but the main rendering thread shouldn't need HDD access. It does a bit of disk IO during the initialization, but not after that.

Share this post


Link to post
Share on other sites
It depends on the Threading type that is used by the operating
system or library you are using.

There are 2 types of Threads:
Kernel-Level Threads and User-Level Threads.

i do not know how you access the harddisk but i think you use something
like fopen or fstream. these calls are blocking system calls so if
the created thread is a "User-Level" Thread the whole Process will have to
wait until the read function returns.

All blocking system calls are bad for "User-Level" Threads.

Kernel-Level implementations should not have such problems tough.(afaik)

Maybe you should lower the priority for the texture loading threads.

For more information:
http://en.wikipedia.org/wiki/Thread_%28computer_science%29

Share this post


Link to post
Share on other sites
I'm using pthreads on linux and OSX, which I think are kernel level. On Windows I use Windows threads (CreateThread, etc).

You are right that I'm using fopen/fstream, and it may look like these calls are blocking the process, so this explanation sounds plausible, even though I'm pretty sure I use kernel level threads.

I've tried lowering the priority of the data reader thread, but that just caused the reader thread to stop entirely.

Share this post


Link to post
Share on other sites
Maybe the rendering thread is starving, waiting for texture data to become available, refusing to render meanwhile the data is not ready... just a thought.

Share this post


Link to post
Share on other sites
No, it's not that. The render thread is rendering data even if we have long data loading queues, it just stutters terribly. It uses a dummy texture while the real data is loading. But it seems that the file reading operations are blocking the entire process.

Share this post


Link to post
Share on other sites
That sounds reasonable.I'm not quite sure how to go about it though. Would it help to use lower level file IO calls such as open() and read() instead of fopen() and fread()?

Share this post


Link to post
Share on other sites
I think for async read/write you will have to be platform specific.
so i don´t think you can write code for Linux/Windows that looks the same.

I found a site that describes async io for mainly POSIX 4 systems.

Hope it helps:
http://www.ibm.com/developerworks/linux/library/l-async/

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!