You *can* have bouncing and restitution and whatnot in a speculative physics engine, it just has to be a bit more hack-y. You have to do things like storing bouncing parameters for next frame, re-checking contacts after the fact to ensure actual collision, stuff like that.
Which is fine, because you can't prevent tunneling in O(reasonable), so at some point you have to kind of cheat a little. That said, you have a tunneling problem, or something really close to it, and speculative collision was *made* to prevent tunneling. Just throw a handful of iterations at it (it should only take like 5 or 6) and boom! your issue is solved. Sure, there are disadvantages, but you have to give a little sometimes to get a little.