Ideally, the collision would be against the current character's skeleton pose. In reality I'm still unable to figure out how to make that work.
So, for what is worth, this is how it is looking so far:
I made a bigger scenario to test different angles to collide with. There is a little issue when colliding with slopes from the bottom as the ellipsoid will try to slide and that simply makes no sense. I need to detect when the ellipsoid is sliding against the ceiling and make it react as expected.