Now unfortunately, I still don't have an answer for how to resolve the simulation vs controllability problem - but the first thing I decided is that whatever solution I went with, it needed a solid stable foundation. And that is what I've been looking at for the last two weeks.
If you go back to some of the original block shooting videos and look carefully at some of the blocks in the background that have come "to rest", you'll notice that not many of them do this particularly well. Blocks jiggle and twist restlessly, occasionally defying gravity to convulse into a new orientation. There are many reasons this is a bad thing, but the two mains issues are that is unstable, and it's expensive (as the physics engine is endlessly "solving" all the blocks).
At first, I was sure this was an issue with my physics solver (a Successive Over Relaxation LCP algorithm). I put in heavy damping, added over the top "auto-sleep" checks that would try to ignore the bobble, clamped normals to "up" to counter matrix inversion imprecision, and a bunch of other things. None of those worked, so my next step was to abandon SOR and try a successive impulse solver (the same kind used in Tomb Raider Legend - one of the few titles to use in-house physics these days). This was actually a really useful exercise because it forced me to totally decouple the physics and collision systems really cleanly - to them point where I can mix and match these by changing a single line of config. But in the end, the impulse solver jiggled just as much, and stacked even worse.
At this point, I started dumping out the collision data and noticed that the collision depths were really discontinuous. You see at the time I wrote the collision code, I didn't have a stable solver, so just having shapes which actually hit each other meant the collision routines must be working. I've since realised the importance of really smooth accurate depth and normal information. I went back to the basics and re-introduced every collision primitive one by one, finding all the errors in the collision code and fixing them. Bingo - both of my physics solvers (SOR and impulse) were instantly rock solid. I can now fill a whole swimming pool with blocks, and even though they grind around for a while coming to rest, it's good grinding around, and it does always eventually stop. The difference (once you're looking for it) is profound - it just feels so much more solid and real. I'm really glad I took the time to get this right.
Here's a wee moovies of the new physics solver. I was tempted to upload a few clips of things just sitting there (as that's the exciting bit!), but I figured I'd show everything falling into place instead. As usual, click the pictures for the moovies. This week's "Where's Milkshake" is a bit of a giveaway unfortunately (the red blocks don't help hiding him).
The block stacking still isn't perfect (I can stack about 4 with LCP and 3 with successive impulse), and I'm still looking for a solid autodisable scheme - but I'm now much happier with my physics foundation, and am ready to start tacking the really hard problems of physically interacting with things in the game world.
Cheers!
Got any ideas for puzzles yet? I think there should be a fair amount of "interesting" blocks because simply piling up stuff won't scale well -- balancing a huge stair and seeing it collapse before your eyes, well, would be interesting, but not in every stage of the game :)
Hey what about a freezer gun that causes a block to stick to any adjacent blocks, locking it in place?