I personally use my "main thread" as a worker thread too. Whenever any of my threads has nothing to do (e.g. it has to wait for the results of another thread before it can continue), then they make themselves useful by popping jobs from the job queue and doing some work. I basically have a "WaitFor..." busy loop, that continually checks if the condition has been met to exit, else tries to run a job, else after enough tries with no jobs in the queue it yields or sleeps.

Ah, that's a good point. I could allow the I/O thread to do a work job as well if there are no I/O jobs pending.

Regarding embarrassingly parallel jobs -- in order to move these off to the GPU, you also need the consumers of those jobs to be ok with extremely long latencies. It's not possible to get short CPU->GPU->CPU latencies on PC without destroying overall performance.

There aren't going to be many (if any at all) circumstances where a CPU step depends on a single GPU step, no. I want to keep what I calculate on the GPU on the GPU, so to speak. The only instance I can think of is using the CPU to do narrowphase CCD after the GPU does broadphase pruning for collision detection (ideally, I'd do the narrowphase on the GPU as well, but I can't think of any good GPU-friendly CCD algorithm),

Also, don’t “spawn” threads, awaken them. They should already exist and just be idling in a waiting state, waiting for an event to set them in motion.

And a “wait” state is not a “sleep” state.

Whoops, mixing up my terminology here. Yes, I plan to have these worker threads always running and waiting for jobs to be queued, not spawned with the jobs themselves.

You're assigning task to sequential process. This could make the design of the modules simpler by allowing you to use blocking functions instead of requiring asynchronous functions but this will do nothing to increase the performance of your game.

If you can identify specific areas that benefit from parallelism then it might be easier to have a dedicated thread-pool for that function - as an exercise multi-thread your geometry culling. It's a recursive process so it's an easy first step.

From there you'll have to do some research about what is optimal. My guess would be to split the screen twice, into 4 sections, and assign each section to a thread.

It's been a long time since I worked on multi-threaded rendering but at the time (circa 2000) OGL was painful to work with and D3D was more accessible to multi-threading. OGL context switches are expensive and you can't share them between threads.

It's been a long time since I worked on multi-threaded rendering but at the time (circa 2000) OGL was painful to work with and D3D was more accessible to multi-threading. OGL context switches are expensive and you can't share them between threads.

Personally, I still take the approach of only having one thread actually submit work to the GPU, however, I use many threads to prepare that work -- things like scene traversal, culling, sorting, redundant state-change removal, etc, etc, is all done in your own code (i.e. it doesn't use D3D/GL functions at all), so you can thread it however you like

D3D11 does have the ability to write multi-core code with it in a fairly sane manner, but D3D9 is completely lacking, and yeah, GL is a pain in the butt.
Personally, I still take the approach of only having one thread actually submit work to the GPU, however, I use many threads to prepare that work -- things like scene traversal, culling, sorting, redundant state-change removal, etc, etc, is all done in your own code (i.e. it doesn't use D3D/GL functions at all), so you can thread it however you like

This is the key message here - the multithreaded API in D3D11 is mostly relevant if you don't have a strong separation (like Hodgman) between doing some CPU work and submitting your state and draw calls. If you perform some moderate work in between your various API calls, then it is potentially beneficial to execute some chunks of that work on multiple threads. That's primarily due to parallelizing the CPU work, rather than any kind of speed up at the driver level...

What is important (and my usual advice in this type of design discussion) is to design your code so that you can modularly move back and forth from multi-threaded to single-threaded execution. This is actually not too difficult with D3D11, since they implemented the deferred contexts with more or less the exact same interface as the immediate context - your rendering code doesn't actually know the difference unless it specifically asks via an API call. Once your code is modularized, you can profile and it should be much easier to tweak your specific situation for a target hardware configuration.