there is a closed solution, but it's very long!
ultimately you can reduce the problem to:
( u )
( 1 a b c )( uv ) = ( 0 )
( 0 1 d e )( v ) ( 0 )
( 1 )
which has a very complex closed solution of:
http://www.wolframal...,+solve+for+u,vof which only one of the two top solutions should be the correct one.
a,b,c,d,e being rather complex equatinos involving your coordinate p, and the coordinates of the 4 rectangle points starting with:
( u )
( bx-ax cx-bx dx-ax ax-px )( uv ) = ( 0 )
( by-ay cy-by dy-ay ay-py )( v ) ( 0 )
( 1 )
with rectangle being represented with top-left a, top-right b, bottom-right c, bottom-left d. u running left to right, v running top to bottom