Just thinking out loud.
A point is inside a convex polygon mesh if the point is on the negative side of all the face planes.
You could decompose your model into a bunch of convex groups and check those.
The initial bounding box test is a good idea. You may be able to use a hierarchy of them to quickly approach a solution.