i'm implementing pixel perfect collision, and in order to make the check faster i want to cull the unused part of the 2 sprites colliding.
I need to find the red area in the pictures (the rectangles can be scaled, rotated, screwed), possibly the one from the last 2 pictures(where the rectangle is parallel to one of the 2 rectangles), but i'm a little confused. I have the vertices in world space, but i can't figure a way to correctly test the right intersections(for each way i think it could be done i can think of a corner case in which it is wrong).
Is there some algorithm to perform this check? I wasn't able to find one, despite the fact it should be a well studied/known subject (at least, i thought 2D collision is a well known and already deeply studied subject..)
bounding box parallel to the world space axis.
bounding box parallel to the local X axis of one of the 2 rectangles
Ps: i have the transformation matrices of both the rectangles