(I like slicer4ever's suggestion also and Erik's pre-save idea... Maybe a combined - pregenerated quadtrees to some reasonable depth)
I would probably try to make a type of mask that can skip tests for spans. IE: 25 true, 30 false, 14 true, 19 false, etc..
uint16 + bool breaker combos?
This way you could fly through non-pruned regions pretty fast too - actually probably don't need to prune anything then and should still be very fast.