Sign in to follow this  
Side Winder

Image (Pre-)Processing for Face Detection

Recommended Posts

Hey, I'm currently working on a face detection system using C# and I'm currently working on the optimal image pre-processing techniques. Obviously I want the algorithm to be as efficient and accurate as possible. Currently I'm working on the efficiency side, and since image pre-processing can vastly speed up the process as a whole, I'm putting aside quite a lot of time on it. Anyway, I've so far integrated histogram equalisation but it's taking about 1.4 seconds (according to the C# Stopwatch class - not sure how accurate that is) for a 1600x1200 JPG image. For a large set of images, this is quite a long time and I want to cut it down. Perhaps by resampling or a faster method of histogram equalisation. What would be a good resampling method? I've looked into it and the Lanczos method seems the best, but also quite slow? Bilinear looks to be a good choice though? Tell me if I'm wrong :) For a 120x80 image it took 42 milliseconds. So what other processing techniques might it be worth looking into? I know a lot of it is how the decision making process will work, but I'm looking for general ideas. I've looked into segmentation and thresholding so far; they seem like a good way to get rid of background noise? Thanks.

Share this post


Link to post
Share on other sites
Quote:
Original post by Side Winder
Hey, I'm currently working on a face detection system using C# and I'm currently working on the optimal image pre-processing techniques. Obviously I want the algorithm to be as efficient and accurate as possible. Currently I'm working on the efficiency side, and since image pre-processing can vastly speed up the process as a whole, I'm putting aside quite a lot of time on it.

Anyway, I've so far integrated histogram equalisation but it's taking about 1.4 seconds (according to the C# Stopwatch class - not sure how accurate that is) for a 1600x1200 JPG image. For a large set of images, this is quite a long time and I want to cut it down. Perhaps by resampling or a faster method of histogram equalisation. What would be a good resampling method? I've looked into it and the Lanczos method seems the best, but also quite slow? Bilinear looks to be a good choice though? Tell me if I'm wrong :) For a 120x80 image it took 42 milliseconds.

So what other processing techniques might it be worth looking into? I know a lot of it is how the decision making process will work, but I'm looking for general ideas. I've looked into segmentation and thresholding so far; they seem like a good way to get rid of background noise?


Are you resizing the image just to feed it to a recognition engine? If so, I'd suggest dispensing with anything fancy and simply averaging the high-resolution pixels to yield the low-resolution pixels. I'd also try applying any luminance adjustments (equalization, etc.) to the low-resolution version of the image. Also, consider doing something simpler than histogram equalization, like linear contrast stretching (to a fixed range) or even standardizing by simply subtracting the mean and dividing by the standard deviation.

Share this post


Link to post
Share on other sites
Hi SideWinder.

Are you building a face detector or augmenting and existing one?

Why are you preprocessing the images?

I am doing the same thing right now, also in C#, using the technique described by Viola and Jones.

Share this post


Link to post
Share on other sites
Its hard to recommend any pre-processing techniques without knowing how you intend to do your face detection. Resampling, for example, can only reduce the amount of information, and is likely to act as a low-pass filter - which may help or hinder your chosen technique. Equalization just re-map the same information, it doesnt clear any noise, and doesnt find any shape or depth cues.

Two classes of techniques Ive seen working well for face detection where sigma-shape detection & Support-Vector Machine classifiers. With any kind of good hardware, face detection should be possible in real-time - less than 5ms per image.

I advise not to get caught in early optimization - get your method working, THEN find whats taking too long and fix it. Why waste time optimizing an histogram equalization you might never use?

Share this post


Link to post
Share on other sites
Quote:
Original post by Steadtler
I advise not to get caught in early optimization - get your method working, THEN find whats taking too long and fix it. Why waste time optimizing an histogram equalization you might never use?


Steadtler beat me to the punch; I was waiting until you responded to my questions to see what you were doing first. :) So basically: what he said.

From my own experience (I've built other face detectors before) I've always found preprocessing to be not so great. It can definately improve the true positive rate, but also has an annoying tendency to increase the false positive rate-- it can make an obviously non-face image look very face like. :)

However: if you're doing face recognition (determining the identity of the face you've detected) then the literature seems to indicate that there are some very good reasons to preprocess your training images. I can't speak from first hand experience though.



Share this post


Link to post
Share on other sites
I see. Yeah, I'm using neural networks. What's the problem with histogram equalisation? Is it just that it's slower than other methods and yields similar results? Either way, I'll heed your advice and work on getting the method fully working before thinking about optimising.

Share this post


Link to post
Share on other sites
Quote:
Original post by Side Winder
I see. Yeah, I'm using neural networks. What's the problem with histogram equalisation? Is it just that it's slower than other methods and yields similar results? Either way, I'll heed your advice and work on getting the method fully working before thinking about optimising.


What result do you expect to get out of an histogram equalization in the context of a computer vision program?

Share this post


Link to post
Share on other sites
Quote:
Original post by Side Winder
Well, I was under the impression that it helps with distinguishing features after greyscaling. Makes the greys more varied so when the data is put into the ANN it's got a broader range of input?


Having lots of face samples under different lighting conditions will give your ANN a broad range of inputs. It's not uncommon to use several 1000 positive training samples.

Just out of curiosity: why are you using an ANN?



Share this post


Link to post
Share on other sites
Quote:
Original post by Side Winder
Well, I was under the impression that it helps with distinguishing features after greyscaling. Makes the greys more varied so when the data is put into the ANN it's got a broader range of input?


Actually, I think equalization might help, but because it standardizes the images for lighting. In other words, faces under different lighting should occupy a smaller portion of the input space.

Share this post


Link to post
Share on other sites
Quote:
Original post by willh
Quote:
Original post by Side Winder
Well, I was under the impression that it helps with distinguishing features after greyscaling. Makes the greys more varied so when the data is put into the ANN it's got a broader range of input?


Having lots of face samples under different lighting conditions will give your ANN a broad range of inputs. It's not uncommon to use several 1000 positive training samples.


Assuming that equalization (or whatever pre-processing is being used) is effective at improving performance, then time and data may be economized by not needing such a large development data set.

Share this post


Link to post
Share on other sites
Quote:
Original post by Predictor
Quote:
Original post by willh
Quote:
Original post by Side Winder
Well, I was under the impression that it helps with distinguishing features after greyscaling. Makes the greys more varied so when the data is put into the ANN it's got a broader range of input?


Having lots of face samples under different lighting conditions will give your ANN a broad range of inputs. It's not uncommon to use several 1000 positive training samples.


Assuming that equalization (or whatever pre-processing is being used) is effective at improving performance, then time and data may be economized by not needing such a large development data set.


It's possible, though from my own experience (as I mentioned above) the final result will be to increase the rate of false-positive detections, and slow down the detector.

There are plenty of face sets available, for free, online; getting a large training set is trivial.

The most important pre-processing step is making sure the features in your traing data line up as much as possible. Most people use the eyes as a reference point. This alone will speed up convergence more than anything else.

Share this post


Link to post
Share on other sites
Quote:
Original post by willh
Quote:
Original post by Predictor
Assuming that equalization (or whatever pre-processing is being used) is effective at improving performance, then time and data may be economized by not needing such a large development data set.


There are plenty of face sets available, for free, online; getting a large training set is trivial.


Whether those data sets would apply to any given application is an open question.

Regardless, there are reasons to not not want large training sets, other than the cost of obtaining the data.


Share this post


Link to post
Share on other sites
SideWinder: Here are the results of my face detector. I didn't use any preprocessing on the source image.

To train the detector I used about 2500 frontal face images. Training time was approximately 5 hours. The false-positive rate reported at the end of training was less than 1 in 2,000,000.

The multiple detection hits are from scanning the source at different scales. The two false positives in this image could be eliminated with some post-detection checks.

I still haven't optimized the code yet so the detection time should be able to improve.


Face detection

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this