A good approach for detecting lines in an image?

Question

I've written some code that uses OpenCV libraries to detect white lines painted on grass. I need someone's opinion on the approach I used (as I'm sure there's a much better way than mine). Also, the results I'm getting are not as good as I expected because slight variations in the image require tweaking the parameters (and I need to operate on fixed parameters).

My approach so far:

1. Grab image from webcam (and turn into grayscale obviously)
2. Run it through a threshold filter (using THRESH_TO_ZERO mode, where it zeros out any pixels BELOW the threshold value).
3. blur the image
4. run it through an erosion filter
5. run it through a Canny edge detector
6. finally, take this processed image and find the lines using Probabilistic Hough Transform HoughLinesP

Should I change the sequence of the filters?

P.S. I'm not too concerned about processing power; I'm running the HoughLinesP on the GPU B-)

Also, here is a sample image:

The results I'm getting: with canny WITHOUT canny (slightly tweaked parameters)

Any help or guidance would be appreciated! I just have no idea what to do to improve it!

UPDATE After using a really quick skeleton implementation (with TONS of blur) as per the chosen answer, I got this:

Maxantia

There's a reason why you run the blur after the thresholding? it seems to me more logical to invert this two steps – Pianette

Hi, a quick search has given me the following vision.caltech.edu/malaa/publications/aly08realtime.pdf The corresponding opencv and matlab code is also available... code.google.com/p/caltech-lane-detection – Cesta

@NicolaPezzotti I don't know why I did it that way. I originally didn't use blur, and then I added it. I'll try inverting the order and will get back to you. Thanks anyways! – Maxantia

@Cesta this looks very promising! Thanks for the link! I'll read it and see what I can do... – Maxantia

@NicolaPezzotti after I tried the skeleton approach, I realized that performing the blur before the threshold made the results even better! Thank you! :D – Maxantia

@Maxantia in general is a good practice to apply a low pass filter before the application of a threshold in order to remove high frequency noise (in this case we can consider grass texture as a noise). – Pianette

Answer 1

I would try to use a skeleton representation of the image. The problem with your canny, here, is that it basically results in two lines because of the width of the line.

Then I would apply the Hough transform on it.

Stratum

Excellent! I was using erosion trying to achieve the same results, but I never thought to dynamically apply it (just enough so that only a single line remains)... I will give it a go, and will get back to you. Thanks! – Maxantia

OMG this is EXACTLY WHAT I NEEDED! It's AMAAAAZING! It turns out that I was simply applying an erosion filter, when I needed a few extra steps to obtain the skeleton! Now, I'm getting PERFECT line segments of the line. All I need to do is figure out how to join them into one line... THANK YOU SO MUCH! :D – Maxantia

This may be a dumb question, but one has to apply the canny filter after the skeleton representation, doesn't he? So it is no replacement for the canny edge detector? – Crowell

@Crowell so sorry for the late reply. The skeleton representation leaves you with single-pixel-width "skeletons". So there's no need to run a canny edge detector after that point, because all you're left with are edges. – Maxantia

@JonesV, the link is broken. – Admiralty

@JonesV Can you please help me for this : #39976118 – Ashkhabad

Answer 2

One possible solution is to take all the edge points that you obtain from the canny edge detection and fit a line using linear least sqaures (maybe iterative) on these points. This way you always get a single line that "best fits" the edge points. There is virtually no parametrisation involved with this method.

Dialyze

Well, all the preprocessing (including Canny) have a lot of parameters. One question, though: what difference would it make if I use the least squares method vs. the Hough Line transform? Is there a significant advantage (besides computation power, which isn't important to me) – Maxantia

The advantage of least squares would be getting one single line as output, while the Hough transform might provide multiple lines (which I noticed in your result). Re the sensitivity, I agree, which is why you probably could do with an iterative method wherein outliers are filtered out with every iteration. Alternately, you could perform RANSAC coupled with least squares, or even RANSAC on its own to obtain the result you need. – Dialyze

RANSAC sounds like a good idea, but I'm running out of time for now and I'll probably go with another method. Thanks, though! – Maxantia

Answer 3

I was using Canny for indoor images, but for outdoor I find more suitable the Laplace filter and Sobel filter, than apply Probabilistic Hough line Transform (PHT).

If u want to thicker your lines, you should try the Sobel operator after Laplace and finally the PHT. If your image is too nosy it might get worse.

Vivian

I'm sorry, but what is PHT? Please elaborate and be more specific. :) – Maxantia

Sorry, PHT stands for Probabilistic Hough line Transform. – Vivian

Oh, ok. But how do you get your PHT to grab the lines if your edges are a little rough and curvey (like in my last picture)? – Maxantia

at what stage should the dissolve be applied? You mean right before the PHT? – Maxantia

Yes before PHT, I also enhanced/sharpened the the the with parts of the image. – Vivian

Answer 4

RANSAC algorithm may be a good method. This method is similar to regression or interpolation approaches. You should extract points after using an edge detection(best method is canny for this goal as I think). Then you should find best line. For finding the line passing through several points there are different methods such as linear regression or RANSAC. You can find implementation and notes about RANSAC algorithm in this link.

Note that RANSAC and another useful algorithms for this goal are already implemented in OpenCV (as I know in version 3.2) and in Accord NET (a free library for image processing).

Thearchy

Answer 5

Following your last result (after the skeleton filter), you get many small segments. I think you're in a really good position at that point to implement what's been done in this article:

http://www.cs.ubc.ca/~lowe/papers/aij87.pdf

Basically, they provide tools to regroup different features in an image based on how likely they belong to a same object. So all you'd have to do is feed your results to their algorithm and you'd likely get a single line as a result.

Kilburn