The generation thread is constantly making new files. The playback thread only needs to load files when it starts and then once every few minutes, and when it loads them it makes sure it loads every file available at the time (well, it has a cache limit, but it's pretty large). In other words the playback thread is idle disk-wise 99% of the time and it becomes less and less disk dependent as it runs since more and more files are loaded each time it hits the end of current playback. I thought it could be a disk access problem at first too so I made sure I wasn't making any dumb moves.
Ok, that is more an overview of what you're doing, I still don't understand what you're trying to achieve, and why you think that using the filesystem is the best mechanism.
Have you considered other communication mechanisms, for example using a socket or a pipe? These can act as a way to control the relative speeds of a producer and consumer, as the producer will block once it fills the buffer, and the consumer will block until there is data in the buffer.
Another approach is to utilise the shared address space. You can use SDL's atomics or synchronisation primitives such as mutexes or semaphores to control access to a shared data structure.
It seems like an odd design in a single process for one thread to write the data to the filesystem, and another to read it all back more or less immediately, when there are far more direct ways of moving data between the two threads.Of course, if persisting the data is necessary, then there are different considerations, but I still think you might be able to avoid a round trip to the disk.
I apologize for the lack of posts. I've been hard at work at reworking the system to make it work with less dependency on the filesystem (among other things that were broken).
Basically what I have now is:
- PLAYBACK thread requests information from a global (cross-thread) class (let's call it Timeline)
- GENERATION thread pushes information to the Timeline as it becomes available
- TIMELINE thread takes care of saving and writing data - it reads previously saved data when it seems like it might be requested soon, and it saves cached data when the buffer is near full. (Actually the timeline thread is currently disabled for debugging)
As before, the playback thread can be sped up or slowed down while the generation thread runs at full speed.
The system works fine as long as I keep the playback speed to default, however I start having problems when I run it at higher speeds. The faster it goes the higher the chance of a deadlock on either thread's part. When one thread crashes, the other continues as if nothing had happened, so it's not a regular deadlock where both threads wait for the other to finish.
I've tried with and without mutexes, but this changes nothing. I added in printf() statements before and after every mutex lock/unlock call and the output looks as it should. Then I added printf() statements in the playback thread (the one most likely to freeze) to track down exactly where it freezes but it simply stops after a certain function call. For example:
printf("func1() was called"); fflush(stdout);
// code for func1 goes here
printf("func2() was called"); fflush(stdout);
// code for func2 goes here
I get the func1() confirmation but not the func2() confirmation, meaning the thread stops running at the moment func2() gets called. I have several instances of this scenario all around my code now, so it's not just that particular piece of code. This can happen at any time, and it's always right after a function gets called. It just freezes. Also note how I added fflush() calls after each printf() call to make sure it gets written to the file.
I can't debug this because, well, it doesn't crash so my regular debugger (Dev-C++) doesn't inform me of anything. I tried running GDB from the command line but GDB freezes as well, at the start of the program. It does the usual "new thread at [0xblablabla]" but then it freezes instead of showing gdb> and waiting for a command.
I'd say the stack is messed up or something, however like I said, this only happens when both threads are running at high speeds. The program ran fine for 20 minutes straight if I kept the playback thread running slowly (10 ticks per second). I'm confused. I hope this is some noob mistake because I have no idea how to debug this. I can't post any actual code either because it's a rather large program.