How to find the document edges in various coloured backgrounds using opencv python? [Document Scanning in various backgrounds]

Question

I am currently have a document that needs to be smart scanned.

For that, I need to find proper contours of the document in any background so that I can do a warped perspective projection and detection with that image.

The main issue faced while doing this is that the document edge detects any kind of background.

I have tried to use the function HoughLineP and tried to find contours on the grayscale blurred image passed through canny edge detection until now.

            MORPH = 9
            CANNY = 84
            HOUGH = 25

            IM_HEIGHT, IM_WIDTH, _ = rescaled_image.shape

            # convert the image to grayscale and blur it slightly
            gray = cv2.cvtColor(rescaled_image, cv2.COLOR_BGR2GRAY)
            gray = cv2.GaussianBlur(gray, (7,7), 0)

            #dilate helps to remove potential holes between edge segments
            kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(MORPH,MORPH))
            dilated = cv2.dilate(gray, kernel)

            # find edges and mark them in the output map using the Canny algorithm
            edged = cv2.Canny(dilated, 0, CANNY)
            test_corners = self.get_corners(edged)

            approx_contours = []

    (_, cnts, hierarchy) = cv2.findContours(edged.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
            cnts = sorted(cnts, key=cv2.contourArea, reverse=True)[:5]

            # loop over the contours
            for c in cnts:
                # approximate the contour
                approx = cv2.approxPolyDP(c, 80, True)
                if self.is_valid_contour(approx, IM_WIDTH, IM_HEIGHT):
                    approx_contours.append(approx)
                    break

How to find a proper bounding box around the document via OpenCV code. Any help will be much appreciated. (The document is taken from the camera in any angle and any coloured background.)

Preterhuman

Please suggest what type of image should be taken in input to findcontours – Preterhuman

Are there some rules based on the Image that allow us to predict the bounding box, or not, if not you are looking at an ML application, if yes, then please mention some of the rules, the best method should be Gradient analysis or Graph Cut, but can't say for sure without more information on the dataset. – Impetuosity

No, it is not an ML-based application on some dataset. It needs to directly detect edges of a document placed in any background. – Preterhuman

There are no rules, but it should work on any document, let me add a sample image – Preterhuman

#51927543 had a similar problem, but the page is also curved here, and illumination is different. – Medico

Yes. Lemme check this link too. thanks @Medico – Preterhuman

Answer 1

Following code might help you to detect/segment the page in the image...

import cv2
import matplotlib.pyplot as plt
import numpy as np
image = cv2.imread('test_p.jpg')
image = cv2.imread('test_p.jpg')
print(image.shape)
ori = image.copy()
image = cv2.resize(image, (image.shape[1]//10,image.shape[0]//10))

Resized the image to make the operations more faster so that we can work on realtime..

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (11,11), 0)
edged = cv2.Canny(gray, 75, 200)
print("STEP 1: Edge Detection")
plt.imshow(edged)
plt.show()
cnts = cv2.findContours(edged.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
cnts = sorted(cnts[1], key = cv2.contourArea, reverse = True)[:5]

Here we will consider only first 5 contours from the sorted list based on area Here the size of the gaussian blur is bit sensitive, so chose it accordingly based on the image size. After the above operations image may look like..

for c in cnts:
    ### Approximating the contour
    #Calculates a contour perimeter or a curve length
    peri = cv2.arcLength(c, True)
    approx = cv2.approxPolyDP(c, 0.01 * peri, True)
    # if our approximated contour has four points, then we
    # can assume that we have found our screen
    screenCnt = approx
    if len(approx) == 4:
        screenCnt = approx
        break
    # show the contour (outline) 
    print("STEP 2: Finding Boundary")
cv2.drawContours(image, [screenCnt], -1, (0, 255, 0), 2)
image_e = cv2.resize(image,(image.shape[1],image.shape[0]))
cv2.imwrite('image_edge.jpg',image_e)
plt.imshow(image_e)
plt.show()

Final Image may look like...

Rest of the things may be handled after getting the final image...

Code Reference :- Git Repository

I guess this answer would be helpful...

Caliginous

Yo size of gaussian blur is a sensitive? – Medico

Thanks for the superb answer. It works mostly in all backgrounds. – Preterhuman

this code look a lot like the one at pyimagesearch.com/2014/09/01/… – Garges

Answer 2

There is a similar problem which is called orthographic projection.

Orthographic approaches

Rather than doing, Gaussian blur+morphological operation to get the edge of the document, try to do orthographic projection first and then find contours via your method.

For fining proper bounding box, try some preset values or a reference letter after which an orthographic projection will allow you to compute the height and hence the dimensions of the bounding box.

Medico

Let me give it a try. Thanks – Preterhuman