 |
|
|
|
|
|
||
|
|
||
|
|
||
|
|
|
|
|
|
||
|
|
||
|
|
||
|
|
|
|
|
|||||||||||||||||
Add Forum to Favorites | Send Topic To a Friend | View Forum FAQ | Track this topic Page: «« 1 2 |
Last Thread Next Thread |
| Autonomous RC Car Hack (Day ...) |
|
 |
bzroom Member since: 1/20/2003 From: Seattle, WA, United States |
|||
 Posted - 12/28/2006 2:15:53 AM |
||||
Quote: Man.. that is some funnnny stuff right there. Might be the tobacco talkin. I'm taking a look at that OpenCV stuff. Pretty nice image package though I'm having trouble finding other info besides some basic demo's. Thr33d, have you (or anyone) had any experience with this? |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
 Posted - 12/28/2006 12:16:35 PM |
||||
Quote: I don't - it came to my attention through my cousin who's pretty into the stuff, downloaded it, never got around to coding some stuff up due to... no time :-( I recall being impressed by it though. The other *main* thing (or what seemed important to me) was some sort of depth vision (like a z-buffer visualization) I'd imagine there's some sort of rangefinding scanner you could hook up. Or you might be able to extract depth using two cameras (like we do) This would be a *huge* step to figuring out obstacle avoidance, and navigating terrain, etc. -M |
||||
|
||||
|
bzroom Member since: 1/20/2003 From: Seattle, WA, United States |
|||
|
Posted - 12/28/2006 2:50:44 PM |
||||
| It's a pretty sweet package though its not as out of the box as the Intel site makes it appear. They're site shows a list of features that could be implemented using the provided functions. DirectShow is suggested for decoding images and video streams. There is a stereo vision disparity calculator function included. Which is the backbone of stereo vision. Here are some links I've found if anyone is interested. Official OpenCV Links: http://www.intel.com/technology/computing/opencv/index.htm http://sourceforge.net/projects/opencvlibrary http://groups.yahoo.com/group/OpenCV/ Simple OpenCV Examples: http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/cvision.htm Complex OpenCV Examples: http://www.fuzzgun.btinternet.co.uk/rodney/components.htm#Motion Stereo Vision: http://www.fuzzgun.btinternet.co.uk/rodney/vision.htm http://www.fuzzgun.btinternet.co.uk/sentience/sentience.htm Direct Show: http://msdn2.microsoft.com/en-us/library/ms787876.aspx http://www.codeproject.com/directx/DirectShowVMR9.asp?df=100&forumid=141179&exp=0&select=1381547 |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 12/28/2006 3:39:48 PM |
||||
Quote: Heh, I just checked back here... yeah, his implementation is a lot better than mine so far ;-)   I'm not taking advantage of edges and continuous sections as well as I should be... -M |
||||
|
||||
|
bzroom Member since: 1/20/2003 From: Seattle, WA, United States |
|||
|
Posted - 12/28/2006 4:42:12 PM |
||||
| Your images are much higher resolution than his. Your results look pretty decent. Much better than my zero results. If you would like to share any of your information with me I am willing :). kincaid05 AT gmail DOT com (<-- first time I've ever done that). |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 12/29/2006 1:42:51 PM |
||||
| Cool beans I'll e-mail with my current method, and my ideas for improvement. Right now it's all in a rapid development environment; I haven't done much with c/c++ lately. We can always convert it over in the end and optimize it. It's slightly too slow right now (I'm assumeing we want ~10fps @ 160x120 resolution as a minimum) most of the slowdown is in massaging the data. I need to widen the recognized data set. I'm planning to revamp the entire thing - are you planning to use color cameras? The next iteration is going to disect the input into sections based upon hue. If all we have is grayscale, we can use that instead, but we won't have the luxery of using the section information to "recognize" or at least "lock on" object since lighting will cut up the sections :-( Let me know what you think - I'll e-mail you with particulars about the last iteration. -M |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 12/29/2006 3:52:14 PM |
||||
| Update - Dang, went to try to ascertain more information using the hue channel (and also to implement object/section indentification) and the hue channel is compressed to crud. The saturation channel had much more information, but unfortunately won't be as clear of a marker as hue *would* be (if the data were better) Here's an example of what I'm talking about: This is the hue channel from that image I was working with earlier:  (this is both left and right images) Sure, the original image is predominantly grayscale to begin with... but I'm worried that the webcam won't have that great of hue sensitivity anyway. It'd be great to get some sample input, so I can play with and find what will work best for breaking the scene into sections. -M p.s. Double update - messing with YUV colorspace as well. If it works out well, it'd be nice since it's a straightforward (matrix) conversion. Downside - it leaves me with 2 channels to deal with (which isn't *too* bad) but I'll need a different distance metric... or I'll have to deal with them seperately. On a seperate note, after getting usable data, I'll look to optimize a histogram, and grab sections using that... I then plan to analyze the image one section at a time, one *line* (or span, if you rather) at a time, figuring that horizontally, for one "object" the offset shouldn't change much across the image. (a noteable exception to this would be driving along a wall... but then, if the wall is relatively flat colored anyway, I doubt the image will be able to give you much good distance information anyway...) After this I might focus on "tagging" objects. We can then begin "tracking" them, as well as throwing some math over the top to estimate actual distance from the camera. This should provide enough input for the control system to "follow" a particular object on-screen (as one of the goals was) ... We'll see how it goes... [Edited by - Thr33d on December 29, 2006 3:52:12 PM] |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 12/29/2006 4:30:25 PM |
||||
| YUV colorspace update: here's that same image With the Y channel (luminance)  W/o the Y channel ("colors" only)  Call me crazy, but that information looks *much* more useful to me that the hue channel from before. It also has the benefit of flattening out the colors into a... much more intuitive distance/similarity metric. I'll try some binning next, and see if we can't begin "tagging" objects on screen. -M p.s. Yes. I did just display yuv information in rgb channels. I think it aids in visualizing them better... Update: Began messing around with UV selection stuff, I'm convinced that YUV is (most likely) the way to go for most situations. Hue just doesn't differentiate enough. Screenshots to follow if interest... btw, should I quit posting here, and just e-mail you directly? Or is anyone else following the development? A binning I shall go... [Edited by - Thr33d on December 29, 2006 6:30:15 PM] |
||||
|
||||
|
bzroom Member since: 1/20/2003 From: Seattle, WA, United States |
|||
|
Posted - 1/3/2007 1:24:50 PM |
||||
| I apologize Thr33d, I kind of fell off the map for a minute there. My develop machine got a nasty virus, I still have yet to get my quick cam's drivers to detect my hardware, and I am also having some pretty aweful health and personal issues right now :(. I really appreciate your interest though I failed to receive your email. I plan to get my DirectX and OpenCV re-installed by today. Please re-contact me either at: kincaid05 AT Gmail or mkincaid AT Vptag Thank you. Please put "GameDev in the start of the subject so my spam filter dosnt block you. |
||||
|
||||
|
AcePilot Member since: 4/1/2005 From: Canada |
|||
|
Posted - 1/4/2007 10:46:44 PM |
||||
| Instead of using a GPS, why not use an accelerometer and gyroscope to track the robot's movements and use them to update the current location of the robot? |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 1/5/2007 12:52:16 PM |
||||
Quote: No problem, hey - I started out in a bit of a sprint (since the robotic vision stuff really interests me, and I love to code up little prototypes) but the new term is coming up (Monday) and I've a rather full plate. Slowing it down to a job is probably a good thing. I'm thinking we carry this on and get it done piece by piece. If I provide the pieces (core functionality) could you write the program that connects them all together? I'll e-mail you in a bit and work out what we can do. By the way, I've updated the stereoscopic part, it still needs some alignment optimization bits in there... but then it'll be about ready for prime time. (don't get me wrong - it's not "incredible" but it is/will be at least on par with the other projects from the intel library we saw.) I'm leaning toward using the stereoscopic part spareingly, since it's rather slow. Object tracking using color information looks to be the optimal solution. Anyway, I'll e-mail it off. Hey - is anyone else interested/following this? Let us know! (edit: sorry, not trying to whore, just thought it would be a shame to continue dev w/o the community on this project) -Michael g. [Edited by - Thr33d on January 5, 2007 12:52:29 PM] |
||||
|
||||
|
Funkymunky Member since: 7/29/1999 From: Exton, PA, United States |
|||
|
Posted - 1/5/2007 1:29:44 PM |
||||
Quote: Hell yeah, I for one have been reading it everytime it gets bumped up |
||||
|
||||
|
Moe Member since: 1/24/2000 From: Calgary, Canada |
|||
 Posted - 1/5/2007 2:44:50 PM |
||||
Quote: Ditto for me too! Moe's blog/devlog | Home of Todo 3.0 | Future home of Todo 4.0 |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 1/5/2007 3:04:02 PM |
||||
| Update: here's the hsl/yuv seperating stuff, along with an example of selecting on a "key" UV value. This shows a comparison of the hsl and yuv decompositions. YUV wins in my book.  Here's some key value selecting of the UV channels  I have an update to the depthmap stuff, which I'll post in a bit. -M |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 1/5/2007 3:22:35 PM |
||||
| Depth perception pictures update, as promised. (edit: added the original for reference) Review - this was the first iteration:  This is a slower, block matching version:  And here's the same algo, working on a 1/2 resolution version of it  (browser should upsize it for you for direct comparison) Dang... broke the hierarchical one, I'll try to post if I can get it working better. I'm toying with an idea of using the UV channels to break the image into different sections which would then be analyzed one horizontal line at a time. This may give similar results at a slightly faster speed, and would scale well for calculating object size from higher resolution input. -M [Edited by - Thr33d on January 5, 2007 4:22:50 PM] |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 1/8/2007 3:25:41 PM |
||||
| Update: Got a digital video camera from the guy around the corner from me. Looked for drivers but didn't find any. I'll try to get drivers from him later today or tomorrow (if there's time). [strike]Has anyone done any video-in programming? Is there a standard interface I can hook up to?[/strike] I'm prototyping this out in a RAD environment, so final performance should be better - but I do need to convert it over. I've been fiddling around with C# lately, perhaps I'll code it up in there :) Quote: Heh - I finally got around to reading the papers you mentioned. The Color Image Segmentation one looked very good, and, strangely enough, is exactly what I've been developing in my mind as I go along :D The Monocular Vision one looked cool because of it's physical use (it's already been done and proven to work) however they were operating in a very specific environment. The implementation we use will have to be a bit more robust. The line scanning idea they use though was what I had in mind for checking for obstacles. I was thinking of using the depth information instead, taking some stats from the line; using y axis derivative information, taking std dev, and average slope over sections. This would allow us to reliably detect when objects larger than the vehicle's wheels were ahead. Scanning precision may not be better than standard curb size for 2ft - 6ft... maybe 12ft provided a well lit (and thus less "noisy"/"grainy" source image). Pictures to come within the next day or two. I'm kind of nervous about the whole video stream stuff, I hope I can find something fast enough and simple enough/problem free enough that it doesn't bog me down too much. Again, if you have a working standard video-in stream type application I could use as a base - or know of some standards out there I could use, please let me know! Classes started today, we'll see how coding time works out. Matt - e-mail me back if you got the messages I sent you - I'm thinking discussion of how to implement stuff my be best done offline, then just updating with project info here. -M edit: I still welcome suggestions for video-in, but it appears the OpenCV stuff may do the trick. In fact, I just found a C# wrapper for the OpenCV library on the yahoo groups discussion board. I also found another person's steroscopic rc truck project. He used the algorithm outlined in the paper linked in the above link. It's actually *very* similar to what I'm using. They set up a similarity threshold which must be met though. It also seems like they throw out the segment if the top 3 matches are too close is similarity (yet distanced apart) since this may be a textured area where the distance cannot be accurately computed. The method I'm using lends itself very well to this and I may implement it. The other improvement I had in mind - matching distances of "blobs" edges - won't do so well mixed in with this method. We'll have to make a decision on which one we want. In the meantime, I'll be trying to get C# to play nice with openCV and get drivers for the camera. [Edited by - Thr33d on January 8, 2007 5:25:58 PM] |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 1/8/2007 8:13:07 PM |
||||
| Update: There is a wrapper for OpenCV in C#, it's called SharperCV. Unfortunately, it's no longer supported nor developed. Apparently it's not difficult to wrap the sucker, but (for instance) the error handling will always be nasty under the cover. On the positive side, however, the reason they abandoned their effort was that Microsoft has come out with their Robotics Studio package which is more extensible than SharperCV. I've downloaded it, and will try to get the camera to show up and begin processing data from there on. -M |
||||
|
||||
|
bzroom Member since: 1/20/2003 From: Seattle, WA, United States |
|||
|
Posted - 1/9/2007 3:05:57 AM |
||||
| Alright, sorry for taking so long. Thr33d I still have not received your messages, but your work is amazing, keep it coming. I completed a different buggy with completely proportional controls. I was looking for a speed controller and I found an old RC car so the old one got the boot (what a waste of money). Fabbed it up in a day, see image below. I also created a little interface program so I can finally start programming vision and obstacly avoidance. I used OpenCV with DirectX to stream the camera. Right now the image is only converted to YUV color space. Every frame this function gets called by a DirectX Filter.
void process(void* img)
{
IplImage* image = reinterpret_cast<IplImage*>(img);
cvCvtColor( image, image, CV_RGB2Luv);
//Process image
}
My next step is to use the information from the previously mentioned papers to implement segmentation, then blob extraction. Then the scan lines aproach to find objects of importance. I plan to first implement a simple "orange ball following" algorithm, just to test my segmentation. Here is a screen shot of my VB app, the rover is controlled through a serial port object, while all the directX and OpenCV takes place in a c++ dll. I couldn't get the 2nd VMR filter to work properly thats why the second video preview looks like its in another window (it is). For the rover I plan on getting a large usb extension and putting a 4 port hub on it to reduce the tether size. Eventually I hope to get belkin's wireless usb hub if i can ever find out where to get it. If not maybe a bluetooth camera and a bluetooth serial port. In the second picture you will notice alot of extra holes, those are for the gps and the BS2 microprocessor which isnt being used right now. I'm simply controlling the servos with a 16 port serial (usb) servo controller from Parallax. Interface:  Rover (made from a "Willy"):  Image with the "top popped":  |
||||
|
||||
|
deathtrap Member since: 2/28/2003 From: Ottawa, Canada |
|||
|
Posted - 1/9/2007 3:40:04 AM |
||||
| Just wanted to say that this is really interesting work you guys are doing, I'm looking forward to future updates, especially on your depth perception problem. |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 1/9/2007 1:16:27 PM |
||||
Quote: @ Honayboyz - do you know yet whether you'll want to use 1 or 2 cameras in the end result? Oh, btw, I think the guy around the corner has that exact same camera, but didn't have drivers that worked in xp for it - if you could hook me up with the drivers - and if you want to copy me on the code you're using for this project, I could (hopefully) quickly and easily plug my code in and see how it does :) (you can pm me or reply here) -M |
||||
|
||||
|
bzroom Member since: 1/20/2003 From: Seattle, WA, United States |
|||
|
Posted - 1/9/2007 7:36:35 PM |
||||
| Thr33d, I want to eventually use two to come up with a depth component, but I'd like to first experiment with one as a fall back. What ever I do with two I'm going to need to be able to do with one first. I'll email you my project to this point. Anyone else intereseted? I'm working on color coding (segmenting?) the pixels in the feed. I'll get back with that soon. |
||||
|
||||
|
Thr33d Member since: 6/24/2000 From: Portland, OR, United States |
|||
|
Posted - 1/10/2007 4:22:23 PM |
||||
Quote: I've pm'ed you my e-mail address. "What ever I do with two I'm going to need to be able to do with one first." Ah, you mean processing - not robot functionality, right? The depth component will give an entirely different set of data than the color segmentation one. We can try to add some logic so the "robot" recognizes "ground color" and can avoid obstacles that way. But when the ground changes color, it's going to be trial and error in detecting flat surfaces from color blobs (for the most part) and that's where the depth perception will come in handy. Please do e-mail me the project. I also echo the anyone else interested question. Classes just started; I'm fighting for getting this project rather finished. The depth stuff still needs (as I previously mentioned) some initial calibrating functionality, to get the images to work with the algorithm currently implemented. I'll also a "certainity" value associated with each depth pixel. We can threshold this later on and fill into the "no depth information" areas. This should (theoretically) give good results in most any situation (crosses fingers) now to test the sucker :) -M p.s. Please copy me on the drivers info, if you can, I'll go grab that other camera and try to get it working (when I have time - probably Saturday) |
||||
|
||||
|
bzroom Member since: 1/20/2003 From: Seattle, WA, United States |
|||
|
Posted - 1/11/2007 6:22:18 PM |
||||
| Thr33d, I emailed you the project, your going to need a lot of pre-req downloads first, they are identified in the email. Sorry it took so long, I wanted to give a good crack at it before I gave it away. But I can't wait to see what you can make of it. I tried various methods. First I had trouble verifying that my LUV space was generated properly. First I zero'ed out the L component to just look at color, then by my understanding the U and V stand for blue and red color components. So I held a blue object and only enabled the U channel. This seemed to generate a grey scale image with the blue object brighter (as expected). But when I tried the same test with a red object and the red channel (V). I did not get good results (could have been because it was a shiny object and was nearly white on screen). I then tried color masking based on the UV values, I converted an RGB value to LUV, then only compared the UV's with some threshold. It gave me entertaining results but not necesarily good ones. It appeared to actualy be thresholding on light intensity, not color. Next I tried, Pyramid Segmentation on the RGB channels. This gave nice looking results but returned way to many connected segments to be useful. Though I was pointing the camera at a rather complex scene. Next I installed cvBlobsLib and ran this over the pyramidal segmented image. The results took days to complete so I scrapped this idea. My last attempt first desampled the image to 70x45, converted to gray scale, then blobslib'ed, this gave me rather nice results, around 20 blobs from the scene. After filtering by area I was left with 5-10 blobs of importance. The problem with this library is the blobs fill method is so slow that if you actually want to see the blobs it will cut your frames by 25%. Anyways, get back when you have some results. p.s Thr33d I forgot to mention that you'll need the cvblobslib library in my email. You shouldnt need it unless you uncomment my blobs code. Also when I was reading about the cvBlobsLib I read that the native cv funtion find contours was actually a faster similar method. I'll have to check that out. p.p.s. This stuff is way harder then I thought. Expecially when you can't debug the code. Since the process function gets called from a thread created by the directx filter. (or am I mistaken, is there a way to do this? it always has the hollow break point) |
||||
|
||||
|
Page: «« 1 2 All times are ET (US) |
Last Thread Next Thread |
|
