Imagine an n-sided polygon, does anyone know a general formula I can apply to detect if an object is within the polygon''s boundaries or outside?
What I intend to use it for is to detect whether my mouse cursor is within a polygon or not.
1. calculate a plane for each egde of the polygon and test the point against them.
2. imagine a point where you know it IS outside of the polygon. then draw a line from the point you want to check to the point which lies outside the polygon. if this line crossed one edge the point is inside the polygon. if it crossed two edges the point is outside the polygon. if it crossed three edges the point is inside the polygon and so on. this also works for non convex polygons. i read this at gamasutra.