In some cases, [volatile] literally does nothing. In other cases, it doesn't do nearly enough. Most of the time, volatile just causes needless de-optimization without solving any real problems.

You need more complex memory barriers to ensure proper ordering of reads and writes (especially since either the compiler or the CPU itself can do instruction reordering or memory store/fetch reordering). For some types of values or some CPU architectures, you also need explicit atomic instructions just to ensure that other threads don't see half of a variable change (not a problem most of the time on x86, but it can be on other architectures).

The important point for me was not why I shouldn't use volatile, but why I didn't need to. I didn't understand that function calls act as memory barriers, ensuring that any variables cached in registers are made consistent with memory at that point.

I am using SDL, at least for the PC versions, but I've had some problems with the Android version and will probably use Android's API directly for that. It advises strict caution when using its atomic support.

What I read implies that POSIX demands most function calls act as memory barriers. If the called function's definition is in a separate source file the compiler can't know its content and whether it in turn locks or unlocks a mutex so I think the only safe option is to assume it might and act in a thread-safe way.

In a single threaded program, whether you write "data=5; readyToReadData=true;" or "readyToReadData=true; data=5;" doesn't matter.
But in a multi-threaded program, this ordering really can matter -- the latter bit of code might mean that another thread sees the Boolean as true, and tries to read the "data" variable before the number "5" has actually been written to it.

Even if your compiler is nice enough to not reorganize your code, you still need to tell the CPU that it's not allowed to reorganize these crucial two steps -- the data must reach RAM before the Boolean does.
To ensure the CPU doesn't mess this ordering up, you need to use (expensive) memory barrier instructions at the right places. The compiler won't do this automatically for every function call because it would make your code 10-100x slower! It only emits these instructions where you ask it to - either automatically when you use mutexes/etc, or manually.

The ordering doesn't actually matter if you use a mutex properly. What I'm unsure of now is how to tell the compiler to insert a memory barrier. A mutex lock/unlock should do that, but what if you're not using the compiler's idea of a native threading API? Eg SDL. I don't see any special attributes in SDL_mutex.h etc. Do programmers have to manually insert a compiler directive, which for some reason I haven't heard of in any threading tutorials?

Is inserting a memory barrier for every function call really such a terrible performance hit? The compiler can work out which variables don't need to be synchronised with RAM (local variables which haven't had a reference taken). Those that do would probably have to be saved on the stack instead anyway.

If you use any properly written mutex (SDL included), it will be performing the required barriers for you. Nothing for you to worry about (except ensuring you're always using the mutexes correctly!)

I should think SDL uses the Windows API on Windows and pthreads everywhere else. What concerns me is say I have a block of code like:

SDL_LockMutex(...);

// Alter some variables to be read by another thread

SDL_CondSignal(...); // Tell other thread to use those variables

SDL_UnlockMutex(...);

The compiler doesn't necessarily know in this scope that the SDL functions have to be memory barriers, because they're not part of its "native" threading API, so it can't apply a memory barrier here unless it assumes that all functions make that necessary. Conversely, when it compiles SDL and encounters what it does consider as its native threading API it can't work its way back up the stack to work out what needs protecting, unless it has some clever run-time trickery.

Side track question:

Considering the the data decomposition approach, is it of any benefit to base the task scheduler on a work stealing design pattern in order to

minimize mutexes on the job queue ? And if yes, is it currently used by any game engine ?

cheers