Hi,

In my image-based renderer, I want to implement a cache that prefetches images from a neighbourhood for rapid retrieval.

I have a main render loop which has to run at 90FPS (VR), and another caching thread which is supposed to run in the background. However, when I change my position, new images are loaded by the cache thread and it often spikes cpu usage to the point where my render thread suffers. This happens even though my render loop is not synchronised with the caching thread. If an image is not in cache and it is supposed to be rendered, it just returns an empty one for the current frame. Nonetheless, I can still observe stuttering when this happens. When I move slowly it is fine however.

When I put a sleep(100) inside my caching thread, the stuttering does not occur, but ofcourse the caching is also a lot slower.

Any ideas why this happens and how I can solve this? My main theory now is that the caching thread performs a lot of memory accesses to load the images. As RAM only has one address line, other threads also suffer. But how can I ever get around this?

Hi,

I think you are right about the root cause of the problem.

I have no experience on this topic:

But I am thinking that the caching thread might have to do its work based on a priority list.

Then images which was skipped in one loop gets a high priority in the next. And then some sort of max work load for the cache system.

Another option could be to have the main thread control the cache system so it is only running when the GPU is busy and the CPU might have less to do. I know it kind of kills the idea of concurrent processing, but it might be use full to for instance trigger every processing loop in the cache system.

Another though:

How many threads are accessing the Hard disc? If you have many access active at the same time. The hard disc might spend a lot of time seeking from one file to the other and back again.

Just some thoughts, hope it pushes your brain work in the right direction :-)

/Kim

50 minutes ago, ApochPiQ said:

How does your caching thread know when to stop doing work and idle?

It observes the current position and caches images around that point in concentric circles. I have set a manual threshold so that it caches only e.g. 5 circles. Every time a significant change in position occurs, the cached area moves with it. Some items will already be in cache, others will have to be loaded or deleted.

1 hour ago, kikr4000 said:

Then images which was skipped in one loop gets a high priority in the next. And then some sort of max work load for the cache system.

I see what you mean. The sleep() I put in the caching thread sort of reduces the max work the thread can do in a certain amount of time in a crude manner and it does indeed seem to help. I have also played with thread priorities a bit, but this does not seem to help. I've always thought the main thread would take priority and just push the caching thread to the background, but apparently that's not how it works. This puzzles me... Isn't this the whole point of priorities?

1 hour ago, kikr4000 said:

How many threads are accessing the Hard disc?

Only the caching thread reads jpeg files from disk. This only takes a fraction of the time compared to the decoding of these though.

I don't understand why the caching thread doesn't just yield when set to a lower priority. It is perfectly fine if it caches fewer images this way, but the rendering should never slow down for it. Surely there exists a mechanism for this no?

EDIT: I'm starting to think that thread priority only applies to raw CPU time and that threads compete equally for memory accesses. This sounds counter-intuitive to me, however.

Maybe you need a more advanced management/ caching strategy in which your caching thread and your render thread have as less friction as possible. The first question is 'who owns' the memory? I would say that your render thread needs to own the storage that maintains the images so caching is just a visitor here that potentially could set some kind of flags that tells render-thread that there isnt an image yet so it could pass over that image to the next one. Checking a bit flag is insofar atomic that there couldnt happen a break between writing bytes so you might not need any interlocked ops here.

Caching thread would know need to kind of responsibilities, tagging memory as obsolete and loading images from disk in the background where I would go for some package/ memory mapping approach rather than loading single files.

A second good caching strategy is to encounter an access timestamp and remove data from cache as needed but as few as necessary and from the oldest access upwards. The propability is very high that if you swing your head left, you will far or less move it back to the right. This would also limit the number of cache misses due to heavy move your head arround. This could otherwise cause your cache to load images that have become obsolete in next frame because you already moved away but then returned so need to cache this again