Jump to content

View more

Image of the Day

Working on an auto spawn system. #gamedev #indiedev #screenshotsaturday https://t.co/Mm2kfekz7b
IOTD | Top Screenshots

The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

Sign up now

Augmented Reality

4: Adsense

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
3 replies to this topic

#1 lmbarns   Members   


Posted 11 April 2013 - 05:14 PM

Firstly, I'm trying to learn more about tracking objects in a video stream.


I have the camera rendering on a texture in a game engine. So far I've been thinking I need to screen capture each frame, or every few frames, and look for matching pixels to track the x, y position of. So if you're tracking a white paper on black surface, as you move it around I can track the position and rotation...



Anyone have any insight to the topic? I've seen there are numerous expensive products and API's, I was just thinking it shouldn't be that impossible to track a simple square or rectangle sheet of paper on a contrasting background, and spawn models on it, adjusting them to the rotation/position of the paper as it moves...



Edited by lmbarns, 11 April 2013 - 05:15 PM.

#2 n0name   Members   


Posted 11 April 2013 - 05:59 PM

You must consider what happens if you rotate your camera... what if you move it away or closer to the object you're trying to track... what if lighting changes....

This are  all factors that affect the way pixels look... so just comparing the pixels themselves isn't a good idea.

Usually what you compare is some correlation between the pixels

Also before you start tracking your object you'll need to find it.

Read about SURF, SIFT, BRIEF algorithms ... they might give you some insight...


If you have a static camera and just move the piece of paper ... you should be able to get away by just finding the corners of the paper by using a feature detector

and then on every frame just calculate features on small area arround each paper corner and compare.... 

#3 VladR   Members   


Posted 11 April 2013 - 06:53 PM

You don't need those expensive products.


In my previous job, we have used OpenCV which is free.



Getting it to compile under Linux environment that you can't really break or properly configure is no fun, though - prepare to switch between different versions of gcc during the build, handling different versions of ImageMagick (and no, certainly not the latest, that tend to explicit various bugs in OpenCV).


Should you desire to compile OpenCV with the GPU support, there is even more fun in front of you :-)


Once you get through the incredibly problematic linux compilation, it is easy to use.


You can start looking into SURF algorithm, of which there are lots and lots of papers on the internet.

VladR    My 3rd person action RPG on GreenLight:    http://steamcommunity.com/sharedfiles/filedetails/?id=92951596


#4 lmbarns   Members   


Posted 13 April 2013 - 11:35 AM

Thanks guys. I've found a few implementations with OpenCV and Unity game engine. SURF looks promising.


If I use unity it seems the tracking will have to be via plugin from something much faster like cpp.


Unity has a method WebCamTexture, which I was using to render the material on a cube. Then I was testing out just finding matching pixels using WebCamTexture.GetPixels32() which gives me a byte array of the pixels of the entire texture.


Looping through the pixels I check each for a match with the color I'm looking for. If it finds it, I divide the pixel index by the width of the texture, and am able to get the x, y coordinate where the pixel was found on the texture.


The other thing I was thinking just for matching the colors, if I add several tiny rects that try to find the corners and just check them rather than grabbing the entire array of pixels making up the entire texture, just check small rects.


I'm aiming for just for a static camera and a non moving paper first, then I'll go from there with movement, looks like it can get really insanely complex quickly.


I was testing it on my face and can use the eyedropper to select a pixel then scan for matching pixels and it finds a decent clump, like if I eyedrop the white of my eyes it finds the white on both eyes. Or a piece of skin on my face finds a cluster around it.

Edited by lmbarns, 13 April 2013 - 11:35 AM.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.