How to extract only characters from image?

Question

I have this type of image from that I only want to extract the characters.

After binarization, I am getting this image

img = cv2.imread('the_image.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
thresh = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 11, 9)

Then find contours on this image.

(im2, cnts, _) = cv2.findContours(thresh.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
cnts = sorted(cnts, key=cv2.contourArea, reverse=True)
for contour in cnts[:2000]:
    x, y, w, h = cv2.boundingRect(contour)
    aspect_ratio = h/w
    area = cv2.contourArea(contour)
    cv2.drawContours(img, [contour], -1, (0, 255, 0), 2) 

I am getting

I need a way to filter the contours so that it selects only the characters. So I can find the bounding boxes and extract roi.

I can find contours and filter them based on the size of areas, but the resolution of the source images are not consistent. These images are taken from mobile cameras.

Also as the borders of the boxes are disconnected. I can't accurately detect the boxes.

Edit:

If I deselect boxes which has an aspect ratio less than 0.4. Then it works up to some extent. But I don't know if it will work or not for different resolution of images.

for contour in cnts[:2000]:
    x, y, w, h = cv2.boundingRect(contour)
    aspect_ratio = h/w
    area = cv2.contourArea(contour)

    if aspect_ratio < 0.4:
        continue
    print(aspect_ratio)
    cv2.drawContours(img, [contour], -1, (0, 255, 0), 2)

Earache

I did research on it. But as they can be explained in English. Like, find contours, dilate the straight lines to close them. That's why, I did not include any code. Also, the code will be pretty meaningless. Because it does not work. – Earache

The original image is not like this. I used adaptiveThreshold to convert it from rgb to binary. I also tried close/open operation to try to close those disconnected lines. I need an approach to tackle the problem. – Earache

All of that is useful information for your question, otherwise you might find people just suggest you try those methods. You need to show what you have tried and where you have failed, that way you can get much better answers – Weepy

I agree with you. Let me edit my question. – Earache

I have two possible suggestions. One is to use a line detector (like Hough) to detect the bounding boxes and separate them into smaller images. You can also try to use cv2.CV_RETR_EXTERNAL in the find contour to remove the contours that are outside, however you get a contour surrounding the whole image... and some numbers touch the boxes (or at least it looks like it) and may not work. You can try with cv2.CV_RETR_TREE and go down some hierarchy levels until you get only the numbers – Murphy

none of the digits is connected to the boxes so it should be pretty straight forward to extract the digits as blobs and then classify them. In general you might use an approach that recognizes characters by the presence of certain features so you don't have to rely on separated characters. but any solution to this problem would be too much for StackOverflow. You should do some own research, read papers and get some understanding of OCR for handwriting – Yodle

@Yodle Can you give me some pointers on any papers that closely resemble my problem? I am pretty new to this domain. – Earache

try this blog for a starter neuralnetworksanddeeplearning.com/chap1.html or maybe this presetnation: inside.mines.edu/~whoff/courses/EENG510/projects/2012/… I also highly recommend that you read a few books on OCR. there are several available. – Yodle

@Earache look for MNIST dataset CNNs. There is a coursera of basic Keras for the MNIST problem: coursera.org/lecture/intro-to-deep-learning/… – Clino

@Arka, I still wondering if there is a image processing only solution just to select the numbers inside the boxes. – Clino

@FredGuth I have moved on from trying to select characters. We are using a deep learning approach. That lets us skip a lot of these preprocessing steps. – Earache

@Arka, can you tell me what deep learning approach? – Calvano

Answer 1

Not so difficult...

import cv2

img = cv2.imread('img.jpg')

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.imshow('gray', gray)

ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU)
cv2.imshow('thresh', thresh)

im2, ctrs, hier = cv2.findContours(thresh.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
sorted_ctrs = sorted(ctrs, key=lambda ctr: cv2.boundingRect(ctr)[0])

for i, ctr in enumerate(sorted_ctrs):
    x, y, w, h = cv2.boundingRect(ctr)

    roi = img[y:y + h, x:x + w]

    area = w*h

    if 250 < area < 900:
        rect = cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
        cv2.imshow('rect', rect)

cv2.waitKey(0)

Result

You can tweak the code like you want (here it can save ROI using original image; for eventually OCR recognition you have to save them in binary format - better methods than sorting by area are available)

Source: Extract ROI from image with Python and OpenCV and some of my knowledge.

Just kidding, take a look at my questions/answers.

Afoul