Text detection on Seven Segment Display via Tesseract OCR

Question

The problem that I am running with is to extract the text out of an image and for this I have used Tesseract v3.02. The sample images from which I have to extract text are related to meter readings. Some of them are with solid sheet background and some of them have LED display. I have trained the dataset for solid sheet background and the results are some how effective.

The major problem I have now is the text images with LED/LCD background which are not recognized by Tesseract and due to this the training set isn't generated.

Can anyone guide me to the right direction on how to use Tesseract with the Seven Segment Display(LCD/LED background) or is there any other alternative that I can use instead of Tesseract.

Jaclynjaco

"I have trained the dataset for solid sheet background " .Would you please mind telling, how you achieved this ? – Leicester

@Jaclynjaco have you made any progress on this? I am running into the same problem. – Stutzman

Answer 1

https://github.com/upupnaway/digital-display-character-rec/blob/master/digital_display_ocr.py

Did this using openCV and tesseract and the "letsgodigital" trained data

-steps include edge detection and extracting the display using the largest contour. Then threshold image using otsu or binarization and pass it through pytesseracts image_to_string function.

Ridgeway

Answer 2

This seems like an image preprocessing task. Tesseract would really prefer its images to all be white-on-black text in bitmap format. If you give it something that isn't that, it will do its best to convert it to that format. It is not very smart about how to do this. Using some image manipulation tool (I happen to like imagemagick), you need to make the images more to tesseract's satisfaction. An easy first pass might be to do a small-radius gaussian blur, threshold at a pretty low value (you're trying to keep only black, so 15% seems right), and then invert the image.

The hard part then becomes knowing which preprocessing task to do. If you have metadata telling you what sort of display you're dealing with, great. If not, I suspect you could look at image color histograms to at least figure out whether your text is white-on-black or black-on-color. If these are the only scenarios, white-on-black is always solid background, and black-on-color is always seven-segment-display, then you're done. If not, you'll have to be clever. Good luck, and please let us know what you come up with.

Kwakiutl

https://mcmap.net/q/661042/-7-segment-display-ocr?rq=1 this stackoverflow question has a link to a c script for reading seven-segment independent of OCR. Probably also worth a look. – Kwakiutl

I am using GPUImageLibrary github.com/BradLarson/GPUImage. I did exactly the same as you did. I applied gaussian blur and then instead of inverting I did Sharpened the blurred image and the provided to the gaussian it worked to some extent but for images that I have added on position 4 in question. It fails... what sort of filters should be applied ? – Jaclynjaco

Is it possible to remove the background of LED ? – Jaclynjaco

The difficult thing about the fourth image is that the background brightness decreases from left to right. I was able to solve this using local adaptive thresholding, called in imagemagick by the function -lat. The idea is to average the pixels in the surrounding area and construct a local threshold value that will separate the foreground from the background. If GPUImageLibrary doesn't have that, it shouldn't be too hard to write yourself. It has the added benefit of still working on flat-background images. On that image, a local adaptive threshold of radius 60-80 pixels worked well. – Kwakiutl

Yes you are right, I have applied Gaussaian blur on the image and then applied AdaptiveThreshold the grains or background is removed. – Jaclynjaco

But Now I am facing a strange problem which is wrong recognition of the characters i.e. for instance I have attached 5th image it is recognixed wrong by tesseract as it returns "n n g g g q" => and I can't map it to "0 0 0 3 8 9" because of the repeated "g"... any idea how can I fix this ? – Jaclynjaco

How are you calling tesseract? From the command line, there are two things you can do. First, make sure it knows there is only one line of text by setting pagesegmode to 7. Second, tell it that every character is a digit by including the config file "digits." The command should look like this: tesseract img.png out -psm 7 digits – Kwakiutl

Is this working out ok? I'm worried that Tesseract might still not recognize the split zeroes, but I have a few ideas for how to deal with that, if you need them. – Kwakiutl

I am sorry I didn't replied to you in a while there was some problem I was facing with leptonica installation... Unfortunately all of the above SS images are returned with wrong results... – Jaclynjaco

btw tesseract-ocr-3.02.eng.tar.gz is what I am using... is there anyother for SS images ? – Jaclynjaco

There's not another tesseract install that'll work better, no. Though there's always the c script for reading SSD I mentioned earlier. What are you doing that is giving wrong results for everything? – Kwakiutl

didnt do anything special... this is my current installation tny.cz/a23ea9ff and I run the command tesseract 000389.png out -psm 7 digits and output is 333339... which is strange and for 004200.png output is 55 333 – Jaclynjaco

chat.stackoverflow.com/rooms/34599/tesseract please join so we can chat there... – Jaclynjaco

Answer 3

Take a look at this project:

https://github.com/arturaugusto/display_ocr

There you can download the trained data for 7 segments font and a python script with some pre-process capabilities.

Cling