Hi there,
i am writing a high performance media player right now and want to implement a fast and stable circular concurrent queue used for adding packets to and getting packets from (single producer, multiple consumer).
Each entry in the queue is preallocated at startup and will never be re-allocated or be freed.
Normally queues like this replaces the data directly for the given index in the array and the consumer just works with a copy of the data.
In my case, i cannot just replace the data with a new one. I must copy the original network packet data so a decoder thread can operate on a copy and the queue can operate on that copy. But this process takes time - dequeing an item from a circular queue does not.
Here comes the problem:
The memory copy from the dequeed packet takes longer than the deque itself and the write position is already advanced - so a decoder thread starts decoding the package - but the package is not ready net - crash... weird behaviour - asserts fire.
What can i do to ensure that the consumer gets a finished copy of the packet and not a broken one using lock-free methods?
See attachments for my naive implementation in C# (WinForms for Visualization). Language does not matter - the problem can be get on any language.