Adaptive threshold of blurry image

Question

I have a fairly blurry 432x432 image of a Sudoku puzzle that doesn't adaptively threshold well (take the mean over a block size of 5x5 pixels, then subtract 2):

As you can see, the digits are slightly distorted, there are a lot of breakages in them, and a few 5s have fused into 6s and 6s into 8s. Also, there's a ton of noise. To fix the noise, I have to make the image even blurrier using a Gaussian blur. However, even a fairly large Gaussian kernel and adaptive threshold blockSize (21x21, subtract 2) fails to remove all the breakages and fuses the digits together even more:

I've also tried dilating the image after thresholding, which has a similar effect to increasing the blockSize; and sharpening the image, which doesn't do much one way or the other. What else should I try?

Demi

Have you seen this thread? – Bifoliolate

I actually did look at this recently, but for the convexity code (which I didn't bother implementing in the end), not the normalization code. I'll definitely take a look at that now. – Respondent

Can you add non-image processing strategies by assuming a well-formed puzzle? If so you could apply some other rules that don't depend on image quality. For example, for cells where you have low confidence in the OCR result, you can check high-confidence digits in each of the three houses for the cell in question, which may allow you to constrain the possibilities for the low-confidence cells. You also could possibly partially solve the puzzle using high-confidence cells, which would add constraints. – Artificial

@Artificial I think this is a really creative solution. I like the idea of using nearby cells to improve confidence. And I don't think you'd even need to partially solve the puzzle, even with simple DFS solving takes very little time. Then you can simply take the first solution you get and assume it is correct. Throw in some parallel processing where each core takes a different assumed board and you'll have a solution in no time. – Rajput

I'm using OpenCV's HoG and SVM to classify the digits. I'm not aware that it's possible to get confidences from that interface. Even if I could, I'd rather just improve the image quality going into the detection. Cool idea though, and I may consider it if all else fails. – Respondent

Answer 1

A pretty good solution is to use morphological closing to make the brightness uniform and then use a regular (non-adaptive) Otsu threshold:

// Divide the image by its morphologically closed counterpart
Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(19,19));
Mat closed = new Mat();
Imgproc.morphologyEx(image, closed, Imgproc.MORPH_CLOSE, kernel);

image.convertTo(image, CvType.CV_32F); // divide requires floating-point
Core.divide(image, closed, image, 1, CvType.CV_32F);
Core.normalize(image, image, 0, 255, Core.NORM_MINMAX);
image.convertTo(image, CvType.CV_8UC1); // convert back to unsigned int

// Threshold each block (3x3 grid) of the image separately to
// correct for minor differences in contrast across the image.
for (int i = 0; i < 3; i++) {
    for (int j = 0; j < 3; j++) {
        Mat block = image.rowRange(144*i, 144*(i+1)).colRange(144*j, 144*(j+1));
        Imgproc.threshold(block, block, -1, 255, Imgproc.THRESH_BINARY_INV+Imgproc.THRESH_OTSU);
    }
}

Result:

Demi

How can i merge blocks? – Isauraisbel

Answer 2

Take a look at Smoothing Images OpenCV tutorial. Except GaussianBlur there are also medianBlur and bilateralFilter which you can also use to reduce noise. I've got this image from your source image (top right):

Update: And the following image I got after removing small contours:

Update: also you can sharpen image (for example, using Laplacian). Look at this discussion.

Hygienic

Thanks for the tips! This is better than the original but doesn't adequately solve the problem of breakages in the digits. – Respondent

Answer 3

Always apply gaussian for better results.

cvAdaptiveThreshold(original_image, thresh_image, 255,
            CV_ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY, 11, 2);

Oria

Gaussian threshold does not always lead to better results. – Marandamarasca