Image transformation in OpenCV

Question

This question is related to this question: How to remove convexity defects in sudoku square

I was trying to implement nikie's answer in Mathematica to OpenCV-Python. But i am stuck at the final step of procedure.

ie I got the all intersection points in square like below:

Now, i want to transform this into a perfect square of size (450,450) as given below:

(Never mind the brightness difference of two images).

Question: How can i do this in OpenCV-Python? I am using cv2 version.

Haughay

As he suggested in his answer, just use standard transforms (the one you tried to use in your first question) for each individual cell ... – Indicatory

I used warp perspective which used 4 corners to transform. Are you saying to use the same taking 4 points of each cell? isn't it a time consuming process? – Haughay

Are you talking embedded programming? I guess not, since you're using python ... on a desktop machine, no, it should not be noticeable, since it's only 81 cells ... – Indicatory

@AbidRahmanK: I haven't tested that, but I'd expect that the running time of the perspective warp transformation is proportional to the number of output pixels. So if it was fast enough to transform the whole grid, it should be fast enough transform 81 grid cells that are only 1/81 of the grid size. – Menken

Answer 1

Apart from etarion's suggestion, you could also use the remap function. I wrote a quick script to show how you can do this. As you see coding this is really easy in Python. This is the test image:

and this is the result after warping:

And here is the code:

import cv2
from scipy.interpolate import griddata
import numpy as np

grid_x, grid_y = np.mgrid[0:149:150j, 0:149:150j]
destination = np.array([[0,0], [0,49], [0,99], [0,149],
                  [49,0],[49,49],[49,99],[49,149],
                  [99,0],[99,49],[99,99],[99,149],
                  [149,0],[149,49],[149,99],[149,149]])
source = np.array([[22,22], [24,68], [26,116], [25,162],
                  [64,19],[65,64],[65,114],[64,159],
                  [107,16],[108,62],[108,111],[107,157],
                  [151,11],[151,58],[151,107],[151,156]])
grid_z = griddata(destination, source, (grid_x, grid_y), method='cubic')
map_x = np.append([], [ar[:,1] for ar in grid_z]).reshape(150,150)
map_y = np.append([], [ar[:,0] for ar in grid_z]).reshape(150,150)
map_x_32 = map_x.astype('float32')
map_y_32 = map_y.astype('float32')

orig = cv2.imread("tmp.png")
warped = cv2.remap(orig, map_x_32, map_y_32, cv2.INTER_CUBIC)
cv2.imwrite("warped.png", warped)

I suppose you can google and find what griddata does. In short, it does interpolation and here we use it to convert sparse mappings to dense mappings as cv2.remap requires dense mappings. We just need to convert to the values to float32 as OpenCV complains about the float64 type. Please let me know how it goes.

Update: If you don't want to rely on Scipy, one way is to implement the 2d interpolation function in your code, for example, see the source code of griddata in Scipy or a simpler one like this http://inasafe.readthedocs.org/en/latest/_modules/engine/interpolation2d.html which depends only on numpy. Though, I'd suggest to use Scipy or another library for this, though I see why requiring only CV2 and numpy may be better for a case like this. I'd like to hear how your final code solves Sudokus.

Tonytonya

Fine, that was marvelous. This is what i was looking for. I knew i could use cv2.remap, but not how to use. But is it necessary to use scipy? can't it be done with numpy and cv2 only? Also can be the same method used for whole image in a single step(ie without splitting into sub-blocks)? – Haughay

It was done for a part of the image to make it easier for myself typing the coordinates of the red points. In "source" you have the coordinates of the red points you have already detected and in "destination" their coordinates in the regular grid. If you'd shared your code I could show how to do it in the code, no need to split the image into subregions. Reg. no use of Scipy, see the update in the answer. – Tonytonya

I'd like to hear how your final code solves Sudokus. which one do you want? how to solve the sudoku grid? or the opencv version on how to do ocr etc.? (solving sudoku grid is taken from a blog , ocr part etc written by myself) – Haughay

I meant an update from you when the sudoku solver is ready would be nice. If that answered your question, you could reward me by selecting as the answer :-) – Tonytonya

Hi, what is the use of creating grid_x, grid_y in this code? what does they mean? – Haughay

docs.scipy.org/doc/scipy/reference/generated/… – Tonytonya

how did you selected the values for grid_x and grid_y ? Is it resulting image size? – Haughay

yes, read carefully the example in the link above, you get a very good idea how that function works. – Tonytonya

Just wondering how did you get the hard coded values in destination and source? – Continuator

@bakalolo dest is a regular grid, so you can calculate the values. In src you need to detect where the lines cross. For one way to do this, see @nikie's answer from the referenced question. – Husbandman

Answer 2

if you have source points and end points (you only need 4), you can plug them into cv2.getPerspectiveTransform, and use that result in cv2.warpPerspective. Gives you a nice flat result.

Endocardium