**1**

# Finding a projection matrix from a 2D image

Started by chucara, Jan 10 2012 05:53 PM

4 replies to this topic

###
#1
Members - Reputation: **122**

Posted 10 January 2012 - 05:53 PM

Hi,

I have been stuck on this problem for quite some time.. I think it is best described with a picture:

Imagine that this picture is taken of a table. I am able to find the four red dots and their coordinates on the pictures. Same goes for the green dot.

Now, I want to be able to find the location of the green dot in 2D space on the table. Assuming the following mappings:

(171,22) -> (0,0)

(488,155) -> (1,0)

(24,126) -> (0,1)

(361, 346) -> (1,1)

How do I determine what coordinate (167,128) maps to on the plane created by the table?

I seem to have read that I need to find a projection matrix (which I have a basic understanding of what is), but how?

Any hints or suggestions you can give me is very welcome, example calculations even more so.

Also, please point out if I am reinventing the wheel here. I am not aware of any libraries that do this already. (for .NET)

I have been stuck on this problem for quite some time.. I think it is best described with a picture:

Imagine that this picture is taken of a table. I am able to find the four red dots and their coordinates on the pictures. Same goes for the green dot.

Now, I want to be able to find the location of the green dot in 2D space on the table. Assuming the following mappings:

(171,22) -> (0,0)

(488,155) -> (1,0)

(24,126) -> (0,1)

(361, 346) -> (1,1)

How do I determine what coordinate (167,128) maps to on the plane created by the table?

I seem to have read that I need to find a projection matrix (which I have a basic understanding of what is), but how?

Any hints or suggestions you can give me is very welcome, example calculations even more so.

Also, please point out if I am reinventing the wheel here. I am not aware of any libraries that do this already. (for .NET)

###
#2
Crossbones+ - Reputation: **2140**

Posted 11 January 2012 - 08:01 AM

I believe you can solve this problem using bilinear interpolation. To start with you have the formula,

x(s,t) = a_1 * s * t + a_2 * s * (1 - t) + a_3 * (1 - s) * t + a_4 * (1 - s) * (1 - t)

where 0 <= s <= 1, 0 <= t <= 1, and the a_i are 2D coefficients that we need to figure out. I'm going to define the boundary conditions you've provided as

A = (171,22)

B = (488,155)

C = (24,126)

D = (361, 346)

So now we use the boundary conditions to get

x(0,0) = a_4 = A

x(1,0) = a_2 = B

x(0,1) = a_3 = C

x(1,1) = a_1 = D

Or,

x(s,t) = D * s * t + B * s * (1 - t) + C * (1 - s) * t + A * (1 - s) * (1 - t)

This gives us a mapping from [0,1]x[0,1] to the co-ordinates specified by A, B, C, and D, but we actually need to the reverse so we need to invert this equation. First, the equation is actually made up of two equations: one for x component and one for y. We can write this as

P.x = D.x * s * t + B.x * s * (1 - t) + C.x * (1 - s) * t + A.x * (1 - s) * (1 - t)

P.y = D.y * s * t + B.y * s * (1 - t) + C.y * (1 - s) * t + A.y * (1 - s) * (1 - t)

where P is the point that we are trying to map to the [0,1]x[0,1] domain. To do this we isolate the 's' and 't' variables,

P.x = s * (D.x * t + B.x * (1 - t) - C.x - A.x * (1 - t)) + (C.x + A.x * (1 - t))

s = P.x - (C.x + A.x * (1 - t)) / (D.x * t + B.x * (1 - t) - C.x - A.x * (1 - t))

Similarly, for the y equation

s = P.y - (C.y + A.y * (1 - t)) / (D.y * t + B.y * (1 - t) - C.y - A.y * (1 - t))

Equating the equations for 's' we get.

(D.y * t + B.y * (1 - t) - C.y - A.y * (1 - t)) * (D.x * t + B.x * (1 - t) - C.x - A.x * (1 - t)) = (P.x - (C.x + A.x * (1 - t))) * (P.y - (C.y + A.y * (1 - t)))

As messy as this all looks it is just a quadratic equation. Solve for 't' and then plug it back into one of the equations for 's'.

-Josh

x(s,t) = a_1 * s * t + a_2 * s * (1 - t) + a_3 * (1 - s) * t + a_4 * (1 - s) * (1 - t)

where 0 <= s <= 1, 0 <= t <= 1, and the a_i are 2D coefficients that we need to figure out. I'm going to define the boundary conditions you've provided as

A = (171,22)

B = (488,155)

C = (24,126)

D = (361, 346)

So now we use the boundary conditions to get

x(0,0) = a_4 = A

x(1,0) = a_2 = B

x(0,1) = a_3 = C

x(1,1) = a_1 = D

Or,

x(s,t) = D * s * t + B * s * (1 - t) + C * (1 - s) * t + A * (1 - s) * (1 - t)

This gives us a mapping from [0,1]x[0,1] to the co-ordinates specified by A, B, C, and D, but we actually need to the reverse so we need to invert this equation. First, the equation is actually made up of two equations: one for x component and one for y. We can write this as

P.x = D.x * s * t + B.x * s * (1 - t) + C.x * (1 - s) * t + A.x * (1 - s) * (1 - t)

P.y = D.y * s * t + B.y * s * (1 - t) + C.y * (1 - s) * t + A.y * (1 - s) * (1 - t)

where P is the point that we are trying to map to the [0,1]x[0,1] domain. To do this we isolate the 's' and 't' variables,

P.x = s * (D.x * t + B.x * (1 - t) - C.x - A.x * (1 - t)) + (C.x + A.x * (1 - t))

s = P.x - (C.x + A.x * (1 - t)) / (D.x * t + B.x * (1 - t) - C.x - A.x * (1 - t))

Similarly, for the y equation

s = P.y - (C.y + A.y * (1 - t)) / (D.y * t + B.y * (1 - t) - C.y - A.y * (1 - t))

Equating the equations for 's' we get.

(D.y * t + B.y * (1 - t) - C.y - A.y * (1 - t)) * (D.x * t + B.x * (1 - t) - C.x - A.x * (1 - t)) = (P.x - (C.x + A.x * (1 - t))) * (P.y - (C.y + A.y * (1 - t)))

As messy as this all looks it is just a quadratic equation. Solve for 't' and then plug it back into one of the equations for 's'.

-Josh

--www.physicaluncertainty.com

--linkedin

--irc.freenode.net#gdnet

###
#3
Members - Reputation: **122**

Posted 11 January 2012 - 11:35 AM

Josh,

I can't yet claim to understand what you wrote, but I will have a look when I'm near some paper (I think I will need it). If this works, you are truely a mathemagician

Thanks a lot, I'll be sure to let you know if I can get it running.

I can't yet claim to understand what you wrote, but I will have a look when I'm near some paper (I think I will need it). If this works, you are truely a mathemagician

Thanks a lot, I'll be sure to let you know if I can get it running.

###
#4
Crossbones+ - Reputation: **2140**

Posted 11 January 2012 - 11:49 AM

Josh,

I can't yet claim to understand what you wrote, but I will have a look when I'm near some paper (I think I will need it). If this works, you are truely a mathemagician

Thanks a lot, I'll be sure to let you know if I can get it running.

Well, make sure you check my working because I am very bad at making silly errors along the way.

Hope it works :-)

-Josh

--www.physicaluncertainty.com

--linkedin

--irc.freenode.net#gdnet

###
#5
Members - Reputation: **122**

Posted 11 January 2012 - 04:39 PM

I have taken a look at it, and it does indeed seem like a correct way of doing this. However, I'm having a little trouble isolating t in the final equation without substituting the values for the ones that are known.

This means that I can at least verify that this is a correct solution by hand, but I can't implement it in code just yet.

Once again thanks. I will verify the solution tomorrow as it is getting late.

This means that I can at least verify that this is a correct solution by hand, but I can't implement it in code just yet.

Once again thanks. I will verify the solution tomorrow as it is getting late.