Assuming two lines in 3D space that may or may not intersect (definied by points of origin and direction vectors, really a ray), how do I determine what the closest point on line A to line B is? I understand how to find the closest point on a line to a discrete point, but having to solve for an unknown point on a line has me stumped.

If you define your lines as follows (parameterized vector functions):

A(s) = PA1 + (PA2 - PA1)s
B(t) = PB1 + (PB2 - PB1)t

Then the distance between your two lines is:

D(s, t) = B(t) - A(s)

From calculating the distance of a point to a line, we know that the closest point occurs when the vector from the point on the line to the point in space is perpendicular to the direction of the line.

Similarly, this must be the case for two lines, too. However, it must be maintained for *both* lines.

Let La = (PA2 - PA1),
Lb = (PB2 - PB1)

Thus, La . D(s, t) = 0, and
Lb . D(s, t) = 0

Substituting, we get:

La . (B(t) - A(s)) = 0, and
Lb . (B(t) - A(s)) = 0

Expanding, we get:

La . ( (PB1 - PA1) + Lb * t - La * s) ) = 0, and
Lb . ( (PB1 - PA1) + Lb * t - La * s) ) = 0

Expanding again, we get:

La . (PB1 - PA1) + La . Lb * t - La . La * s = 0, and
Lb . (PB1 - PA1) + Lb . Lb * t - Lb . La * s = 0

Finally, rearranging the equation, we end up with:

(La . La)s - (La . Lb)t = La . (PB1 - PA1), and
(La . Lb)s - (Lb . Lb)t = Lb . (PB1 - PA1)

This is a system of equations in s and t that should be easy enough to solve.