Detecting truck wheels

Question

I am currently working on a project which we have a set of photos of trucks going by a camera. I need to detect what type of truck it is (how many wheels it has). So I am using EMGU to try to detect this.

Problem I have is I cannot seem to be able to detect the wheels using EMGU's HoughCircle detection, it doesn't detect all the wheels and will also detect random circles in the foliage.

So I don't know what I should try next, I tried implementing SURF algo to match wheels between them but this does not seem to work either since they aren't exactly the same, is there a way I could implement a "loose" SURF algo?

This is what I start with.

This is what I get after the Hough Circle detection. Many erroneous detections, has some are not even close to having a circle and the back wheels are detected as a single one for some reason.

Would it be possible to either confirm that the detected circle are actually wheels using SURF and matching them between themselves? I am a bit lost on what I should do next, any help would be greatly appreciated.

(sorry for the bad English)

UPDATE

Here is what i did. I used blob tracking to be able to find the blob in my set of photos. With this I effectively can locate the moving truck. Then i split the rectangle of the blob in two and take the lower half from there i know i get the zone that should contain the wheels which greatly increases the detection. I will then run a light intensity loose check on the wheels i get. Since they are in general more black i should get a decently low value for those and can discard anything that is too white, 180/255 and up. I also know that my circles radius cannot be greater than half the detection zone divided by half.

Eusebioeusebius

Just by chance - you are aware that your requirement is a major image analysis project? Not "i make a program" but "I write a research paper on it"? So, the question can not be answered here - because either you ask for a library (off topic) or - well - it is WAAAAAYYYY too intensive for a simple q&a. – Bobbybobbye

Yes i am aware, am looking for a response from a Computer vision Guru. I think the answer could be simple. A tweak i could do on a SURF algo to reduce its precision or something along those lines. Theirs alot of more complicated stuff out there that got answers. Thanks for the help tho... – Eusebioeusebius

I don't think you'll find someone who will answer because like @Bobbybobbye said, there's still a lot of research going into this. Here are some hints though. You'll need to experiment with different lenses and find one that behaves well enough for your library. You can do some preprocessing to try to filter out some bad stuff. Tires are always "blackish", put a heavier filter to weigh out your blacks and will drop everything else into white. The library has problems with skew, see if you can deskew. All tires should be roughly the same size, throw away anything that isn't in that range. – Fujio

Thanks Shoe, you have a good idea i will try to implement this and see what i can get. I am not looking for a 100% detection. Just the closest i can get. All those ideas will greatly help in the detection. This is more what am looking for. – Eusebioeusebius

Also, i know SURF algo can detect similar textures even if they are skewed or rotated, is there a sensitivty setting to this algo ? Something like a match score that i could evaluate ? – Eusebioeusebius

I think the solutions presented are a step in the right direction. No doubt a non trivial problem, but depending on your constraints it may be more or less solvable. A few questions 1) Do you have to exclude cars driving on this road from analysis? 2) Does the system have to handle and exclude people walking, or even groups of people? 3) Suppose that two trucks are driving by one tailgaiting the other? Will the system have to distinguish and segment both trucks? 4) How diverse is the set of trucks you need to distinguish? – Santasantacruz

1) Yeah i have random cars passing by, i need to remove them if its the case 2) Yes is does, i am able to do so by using detection motion and then validating the size of what moves 3) Right now it is capable of doing so because of what i detected from motion detection gets goes forward until it exists and then motion is detected in the middle of the photo 4) large template set, from 2 sets of wheel to 8 sets including trailers. – Eusebioeusebius

Answer 1

In this answer I describe an approach that was tested successfully with the following images:

The image processing pipeline begins by either downsampling the input image, or performing a color reduction operation to decrease the amount data (colors) in the image. This creates smaller groups of pixels to work with. I chose to downsample:

The 2nd stage of the pipeline performs a gaussian blur in order to smooth/blur the images:

Next, the images are ready to be thresholded, i.e binarized:

The 4th stage requires executing Hough Circles on the binarized image to locate the wheels:

The final stage of the pipeline would be to draw the circles that were found over the original image:

This approach is not a robust solution. It's meant only to inspire you to continue your search for answers.

I don't do C#, sorry. Good luck!

Gallardo

The moment I saw the first image, I knew it will be you :) – Alcibiades

@AbidRahmanK hahaha! I've been away from SO, there's so much work going on nowadays, I barely have time to come here and answer questions. How is your blog doing? – Gallardo

Me too... got busy with thesis works.. So not caring on blog nowadays.. Even in SO, I answer only simple questions which can be answered within 5 min ... Need to take a one month vacation after college, just to refresh the head...:) – Alcibiades

I would love to see some new content on your blog. It's really awesome! – Gallardo

Answer 2

First, the wheels projections are ellipses and not circles. Second, some background gradient can easily produce circle-like object so there should be no surprise here. The problem with ellipses of course is that they have 5 DOF and not 3DOF as circles. Note thatfive dimensional Hough space becomes impractical. Some generalized Hough transforms can probably solve ellipse problem at the expense of a lot of additional false alarm (FA) circles. To counter FA you have to verify that they really are wheels that belong to a truck and nothing else.

You probably need to start with specifying your problem in terms of objects and backgrounds rather than wheel detection. This is important since objects would create a visual context to detect wheels and background analysis will show how easy would it be to segment a truck (object) on the first place. If camera is static one can use motion to detect background. If background is relatively uniform a gaussian mixture models of its colors may help to eliminate much of it.

Unhappy

+1 Absolutely, if the OP is working with videos is very important to subtract the background first, thus improving the detection of the wheels. – Gallardo

Also if view point is such that long axis of ellipses is vertical one can run 4D Half to find ellipses instead of 5D. – Unhappy

An interesting possibility is detection of rotating wheels. It is like structure from motion but when an object moves rather then when a camera moves. With enough resolution and fps the rotating wheels will probably be the fastest moving objects in a scene not to mention their sinusoidal moving signature. – Unhappy

Answer 3

I strongly suggest using: http://cvlabwww.epfl.ch/~lepetit/papers/hinterstoisser_pami11.pdf

and the C# implementation: https://github.com/dajuric/accord-net-extensions

(take a look at samples)

This algorithm can achieve real-time performance by using more than 2000 templates (20-30 fps) - so you can cover ellipse (projection) and circle shape cases. You can modify hand tracking sample (FastTemplateMatchingDemo)

by putting your own binary templates (make them in Paint :-))

P.S: To suppress false-positives some kind of tracking is also incorporated. The link to the library that I have posted also contains some tracking algortihms like: Discrete Kalman Filter and Particle Filter all with samples!

This library is still under development so there is possibility that something will not work. Please do not hesitate sending me a message.

Subjective

+1 The work seem very interesting, thanks for pointing it out. – Gallardo