I have one simple question. Figure 1 shows that if the line AB is extended, it somewhere intersects the rectangle at C. My question is how to find the intersection point C. I know the coordinates of A and B.

Ray with t as independent variable: R(t) := A + t * ( B - A )

An edge (exemplarily one of the four building the rectangle) with k1 as independent variable: E1(k1) := [-1,-1] + k1 * [1,0]

Then R(t) = E1(k) gives the point of intersection. These are 2 linear equations with 2 independents what can be solved symbolically. The result cannot be computed (or, to be precise, gives infinity of negative infinity) if R(t) is parallel or anti-parallel to E1(k1). Otherwise a specific pair (t',k1') can be computed. To be valid, k1' must be in the interval [0,+1], and t' must be greater than 0. If so, then R(t') and E1(k1) is the solution. Otherwise another edge has to be investigated.

I have one question. In equation R(t) = A + t*(B - A), here t is 0 to 1 in between A and B. But this is something like extrapolation for C. Wil this work for extrapolation? Please clarify a bit.

Ok, then will the value of 't' be greater than 1 in this case?