Decided to rip out the grab and shimmy system and start again because the previous version was buggy and wouldn't cope with corners very well. It is important to me to maintain a smooth velocity as you shimmy from one shape to another and this was impossible with the old system.
The approach now is to use a single point of contact, a fixed offset from the player's position. When the character is jumping or falling, this position is polled to see if it is within an epsilon of an edge and if so, the player is locked to that position and goes into the grab state.
A nominal "width" is defined for the player (0.25f) which is the distance from this point to each hand. When the grab is initiated, the grab subsystem uses the physics broadphase to look up the edges (if any) to the left and right of the current edge and stores these for easy look up.
As the player approaches the ends of the edge, the subsystem looks to see if there is an adjacent edge and locks the player to end - width if not so you can't sort of float with one hand in mid air.
If there is an adjacent edge, if the current shimmy step takes you beyond the end of the current edge, the subsystem figures out how much you went beyond the end of the edge, sets the current edge to the adjacent edge (doing a new left/right adjacent lookup) and projects you along the new edge from the appropriate end by the amount you went over the previous edge.
So as you shimmy from one edge to another, the velocity remains perfectly smooth and constant.
The final touch is that when you are in grab or shimmy mode, your rotation is set to be facing the opposite of the normal of the current edge. The subsystem checks if you are closer than the "width" to the end of the current edge and if so, and if there is an adjacent edge, it takes your current distance from the end and divides it by the "width" to get a t value that can be plugged into an interpolate method, taking the current edge normal, the next edge normal and this t value to get an interpolated direction.
In English, as you approach a join between edges and shimmy past onto the new edge, your facing direction smoothly changes. You can see this in the video as you move around the corners.
Its all organised into its own little subsystem code-wise. State machines are an excellent way to compartmentalise complex character controllers and I highly recommend it as an approach. Grab and shimmy is completely non-dependant on any other parts of the character controller so as I add new behaviours, it won't cause any complication, no matter how complex it is internally.
At the moment, there is no further collision detection or response while in grab and shimmy modes, so you'd shimmy right through a solid shape if it was in your way, but this should be easy enough to deal with. I can probably just take the separation vector returned by the physics subsystem and project it back along the edge normal to get a resolved position here.
Thanks for reading as usual.