Computer Vision, Detecting roads, where to start? [closed]

Question

I am an EE undergrad, I am working on a project that requires me to detect roads (primarily the turns). The requirement is that, given the GPS coordinates and directions of the turns, the robot should be able to navigate it's way to a given point. The problem is that the GPS coordinates are very inaccurate and roads aren't always straight. So I will have to detect the sides of the road and navigate my robot accordingly.

I am thinking mounting two cameras on either sides of the robot. Which would normally be used to keep the robot in the center of the road and when robot is withing 5 meters of the turn, one of the camera would guide it through the turn.

I will be working on raspberry pi, but for testing purposes i have installed simplecv and opencv on my laptop running Ubuntu. I have absolutely no prior experience with computer vision. I have no idea where to start. Could someone please guide me through the algorithm for achieving the above mentioned task? Should I be working with simplecv or opencv? Python or C++? Personally i like the simplecv on python, but I have no idea if it is capable of achieving the task.

Any help would be appreciated. Put me on the right track!

I will be demoing the project on campus, here the pictures of the campus's roads.

Edit: Ideal conditions, No traffic, No obstacles. Constant road width.

Monica

Try reading up on edge detection for sharp turns or HoughLine transform, drawing out the lines of the sides of the roads. Just of the top of my head... But wouldn't normal sensors like IR, ultrasonic, etc be sufficient instead of computer vision? Not a robotic expert here, so just asking – Attenuant

@Attenuant IR sensors have limited range (just a few cms). I tried ultrasonic but the pavement is too low, ultrasonic either detects the road or nothing at all. – Monica

Answer 1

A suggestion, that I think will work, so give it a shot. The 4 images I will be using are in reference from "OpenCV 2 Computer Vision Application Programming Cookbook" by Robert Laganiere. This was one of the books I used for studying image processing, and there was a similar example on road using HoughLine.

I used HoughLine before, but not on roads. So to give u better idea, here it is=>

The original image is as seen:

Now u can apply Canny to the image, and it will look like this:

OR u can use Sobel too...

After which, u apply HoughLine:

You will have to adjust the parameters yourself.

So here is my suggestion=>

Put a front camera, slightly low enough to be able to detect the lanes up to a few metres, you can use ROI too(Region of Interest) to focus on the bottom half of the video where the lanes are most likely to be. This is to eliminate noise, where other objects parallel to the road is detected as well.

You make the robot stay in the centre both the detected road lines. And another usage of the ROI is that one side of the line disappear,means that it is moving sightly slanted. You can adjust the robot to go back right on track.

When there is a turn, you can specify that if the lines(maybe through canny operator) is no longer vertical, do a turn till the lines are vertical again.

Of course, you will have to write two different functions to check whether to do a right turn or a left turn based on the angle of the lines.

This is how I would tackle the problem. My method should have pretty decent results. The only problem you may face is adjusting the parameters of Houghlines and Canny.

(P.S. In viewing your pictures, I notice the side of the road kerb have gaps once in a while. So I would recommend HoughLineP instead of HoughLine where you can specify the maximum gap between each line to consider it a line, if I do remember correctly. If that still doesn't work, you may need to do some pre-processing on the frames of the video.)

Hope you find my method useful. Good Luck. If there is anything else you may need help with, comment on this answer, I try my best to help(:

Attenuant

Pardon my informal english anyway. Kind of in a rush now. Peace. – Attenuant

Would this method work without lanes? – Monica

I was thinking of performing HoughTransform on the kerb actually. It will be tricky, but it can be done or You can also use two cameras on the side you have proposed in the question instead to detect each side of the road (one for each side) also via Hough Transform. You can then angle your camera to portray the kerbs on the top half of the captured frames, and set the ROI on the bottom half. If you think about it, if there is a curve on the road to the right for instance, the left side captured frame will show the HoughLine coming into the ROI(bottom half), as the kerb get closer to it. – Attenuant

Hope you know what I mean, it's a bit difficult to illustrate without a whiteboard. The angle of the camera is important for this. Try visualise it. Or you can try something similar in regard to putting the camera on the side. If you ask me for another method, I can only think of processing by colour space. Let me google if people have done it before and propose another answer shortly so that you have more methods to experiment with. – Attenuant

Answer 2

Since you ask "where to start", I suggest you to start from reading about the problem of road/lane detection. Maybe you can check in your campus library for some literature on the problem.

For example the book DAVIES, E. Roy. Computer and Machine Vision: Theory, Algorithms, Practicalities. Elsevier/Academic Press, 2012, has "CHAPTER 23 In-Vehicle Vision Systems" which describes the basic approach to the problem of "Locating the Roadway" and "Location of Road Markings". The book by Davies has also a commented and updated bibliographic section.

For some more detailed background you can check a "survey" article like the following, which compares different approaches to solve the problem:

MCCALL, Joel C.; TRIVEDI, Mohan M. Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. Intelligent Transportation Systems, IEEE Transactions on, 2006, 7.1: 20-37.

Berth

Answer 3

As promised, here is another method.

To have a clear picture of what this method is about, watch this video: http://www.youtube.com/watch?v=TMKd9ov_rmE

The colour segmentation method is actually using Expectation-Maximization (EM) algorithm. And Gabor filters for vanishing point. Kind of close to the idea I have in mind but slightly different.

You can try out the method shown in the video or/and mine.

What I would actually do is still a vanishing point, and the ROI change according to the y-axis point of the vanishing point. While the sides of the ROI is fixed, such that the road(gray colour is always within the ROI).

After that you have to use area, where once if the colour space of the gray colour in the ROI is less than 90%(u calculate the number yourself. This is just a rough gauge) of the ROI(meaning there is a turn). There are several methods to tell which turn is it, I let you figure out this part yourselves :p. Sure you can think of it if you use this method.

My only concern is the zebra crossing or rather pedestrian in the sample images in white which you have shown, but those can be removed pretty easily. (hint. histogram and if the color of values are those in white range.... or you can use area. if blob is less than this amount of pixels... )

I not sure how computationally expensive this program will be though.

But if you are going to try this method, focus on HSV or LAB colour space instead of RGB. Now you have more methods to experiment from. You can try think of your own methods too. Play around. Image processing, computer vision is really fun (:

Attenuant