How to check if obtained homography matrix is good?

Question

This question was already asked, but I still don't get it. I obtain a homography matrix by calling cv::findHomography from a set of points. I need to check whether it's relevant or not.
The proposed method is to calculate maximum reprojection error for inliers and compare it with a threshold. But after such filtration I keep getting insane transformations with object bounding box transforming to almost a straight line or some strange non-convex quadrangle, with self-intersections etc.
What constraints can be used to check if the homography matrix itself is adequate?

Tiernan

See [this][1] answer I posted some time ago. [1]: #10972938 – Ropable

Answer 1

Your question is mathematical. Given a matrix of 3x3 decide whether it represents a good rigid transformation. It is hard to define what is "good" but here are some clues that can help you

1. Homography should preserve the direction of polygonal points. Design a simple test. points (0,0), (imwidth,0), (width,height), (0,height) represent a quadrilateral with clockwise arranged points. Apply homography on those points and see if they are still clockwise arranged if they become counter clockwise your homography is flipping (mirroring) the image which is sometimes still ok. But if your points are out of order than you have a "bad homography"
2. The homography doesn't change the scale of the object too much. For example if you expect it to shrink or enlarge the image by a factor of up to X, just check this rule. Transform the 4 points (0,0), (imwidth,0), (width-1,height), (0,height) with homography and calculate the area of the quadrilateral (opencv method of calculating area of polygon) if the ratio of areas is too big (or too small), you probably have an error.
3. Good homography is usually uses low values of perspectivity. Typically if the size of the image is ~1000x1000 pixels those values should be ~0.005-0.001. High perspectivity will cause enormous distortions which are probably an error. If you don't know where those values are located read my post: trying to understand the Affine Transform . It explains the affine transform math and the other 2 values are perspective parameters.

I think that if you check the above 3 condition (condition 2 is the most important) you will be able to detect most of the problems. Good luck

Apuleius

In addition to the first suggestion check answers.opencv.org/question/2588/check-if-homography-is-good. Computing the determinant of the what's supposed to be rotation submatrix in the homography and checking whether it's greater or less zero tells you if orientation was preserved (basically computing such determinant is the equivalent of the Pythagorean formula). – Denticulation

Answer 2

Edit: This answer is irrelevant to the question, but the discussion may be helpful for someone who tries to use the matching results for recognition like I did!

This might help someone:

Point2f[] objCorners = { new Point2f(0, 0),
    new Point2f(img1.Cols, 0),
    new Point2f(img1.Cols, img1.Rows),
    new Point2f(0, img1.Rows) };

Point2d[] sceneCorners = MyPerspectiveTransform3(objCorners, homography);
double marginH = img2.Width * 0.1d;
double marginV = img2.Height * 0.1d;
bool homographyOK = isInside(-marginH, -marginV, img2.Width + marginH, img2.Height + marginV, sceneCorners);
if (homographyOK)
    for (int i = 1; i < sceneCorners.Length; i++)
        if (sceneCorners[i - 1].DistanceTo(sceneCorners[i]) < 1)
        {
            homographyOK = false;
            break;
        }
if (homographyOK)
    homographyOK = isConvex(sceneCorners);
if (homographyOK)
    homographyOK = minAngleCheck(sceneCorners, 20d);




     private static bool isInside(dynamic minX, dynamic minY, dynamic maxX, dynamic maxY, dynamic coors)
        {
            foreach (var c in coors)
                if ((c.X < minX) || (c.Y < minY) || (c.X > maxX) || (c.Y > maxY))
                    return false;
            return true;
        }      
        private static bool isLeft(dynamic a, dynamic b, dynamic c)
        {
            return ((b.X - a.X) * (c.Y - a.Y) - (b.Y - a.Y) * (c.X - a.X)) > 0;
        }
        private static bool isConvex<T>(IEnumerable<T> points)
        {
            var lst = points.ToList();
            if (lst.Count > 2)
            {
                bool left = isLeft(lst[0], lst[1], lst[2]);
                lst.Add(lst.First());
                for (int i = 3; i < lst.Count; i++)
                    if (isLeft(lst[i - 2], lst[i - 1], lst[i]) != left)
                        return false;
                return true;
            }
            else
                return false;
        }
        private static bool minAngleCheck<T>(IEnumerable<T> points, double angle_InDegrees)
        {
            //20d * Math.PI / 180d
            var lst = points.ToList();
            if (lst.Count > 2)
            {                
                lst.Add(lst.First());
                for (int i = 2; i < lst.Count; i++)
                {
                    double a1 = angleInDegrees(lst[i - 2], lst[i-1]);
                    double a2 = angleInDegrees(lst[i], lst[i - 1]);
                    double d = Math.Abs(a1 - a2) % 180d;

                    if ((d < angle_InDegrees) || ((180d - d) < angle_InDegrees))
                        return false;
                }
                return true;
            }
            else
                return false;
        }
        private static double angleInDegrees(dynamic v1, dynamic v2)
        {
            return (radianToDegree(Math.Atan2(v1.Y - v2.Y, v1.X - v2.X))) % 360d;
        }
        private static double radianToDegree(double radian)
        {
            var degree = radian * (180d / Math.PI);
            if (degree < 0d)
                degree = 360d + degree;

            return degree;
        }
        static Point2d[] MyPerspectiveTransform3(Point2f[] yourData, Mat transformationMatrix)
        {
            Point2f[] ret = Cv2.PerspectiveTransform(yourData, transformationMatrix);
            return ret.Select(point2fToPoint2d).ToArray();
        }  

Oversubscribe

too much code without comment is only helpful for the desperates – Drawplate

I think function names explain themselves. This is the one I could able to do. Summary: It checks, after transformation by homography, if the found shape is 1) inside an acceptable region, 2) size is acceptable, 3) shape is convex or not(there is a mistake here) 4) shape's inner angles are acceptable or not. There are a few mistakes, I'll update the code when my work in this is over. And yes I am desperate if there is some other way to achieve a better check. I also like being a desperate and read other peoples' codes a lot all the time. – Oversubscribe

FYI: #11053599 is in my opinion the most plausible solution I've seen for this topic. – Drawplate

Thank you for that link. I have read it before. I think it is dependent on the location too much. I don't see how it would be successful for a case like this: emgu.com/wiki/images/StopSignDetectionExample1.png – Oversubscribe

If I understood correctly, you have two distance arrays combining an arbitrary amount of available Points - one with the distances to each other before and one after the perspective transform. These distances (not locations) remain the same if it was a correct determination. In other words having an ideal homography you receive exactly 0 as the difference of both arrays, including real-world cases you need to specify a tolerance range (he defined as < 1 is acceptable). For what I've seen in your picture this should catch your case – Drawplate

I may not understood correctly. But with the image that I have added to my answer, distance would be more than 1 I guess. So this result would be rejected, wouldn't it? I appreciate if you can explain how I am wrong with that; if I am. – Oversubscribe

please excuse I'm working only part time on this issue. You were right the direct distance-comparison between two points before/after homography is indeed only applicable for a small skew value. However what shouldn't be affected is the relation between two distances: d1_before_homography / d2_before_homography != d1_after_homography / d2_after_homography +/- tolerance agreed? – Drawplate

I appreciate you answer. I may not understand something that you see. The stop sign on the left (before homography) has a square shape, say it has 1-1-1-1 unit length each segment. Green lines on right are values after transform seems like 2-2-1-1. Apply formula 1/2==1/2==1/1==1/1 , this is OK for this one for tolerance 1. Is it the same formula on https://mcmap.net/q/500511/-how-can-you-tell-if-a-homography-matrix-is-acceptable-or-not Also the distances may be OK according to this formula, but the result shape could be a concave or self intersecting, which should be rejected. I hope didn't got you wrong and could express myself clearly – Oversubscribe

Yes I think we have a different perspective on the problem: dx_ before_ homography refers to your left planar (i.e.: the ideal comparison) image of the stop sign. dx_ after _ homography isn't shown here at all. It is the already warped image that resembles the planar, (e.g.: inst.eecs.berkeley.edu/~cs194-26/fa15/upload/files/proj7B/… the rectified and cropped one). If you compare the distances here with the one on the original image, you can easily check whether your homography went wild or it is a plausible result. – Drawplate

I've built an inverseHomogrphy matrix and find the rectified values for sceneCorners. These values and distances are almost the same (-+ <0.01) with the original image corner points on almost every images I tried. How ever most of them are wrong and should be rejected. I'm really trying to understand, but I couldn't :( – Oversubscribe

Pardon me for the lack of a nice explication via comments. Simply spoken: (1) you have figured out the keypoints. (2) the homography is calculated (3) generate the rectified warped image (4) use the calculated homography not only to stretch the image, but to find out where the keypoints from (1) are on the image of (3). It is simply just kp_bef * homography = kp_aft (I'm not sure about the order of matricies and the inverse homography here). (5) Compare these KP and check if the distances in(1) are alike to the ones in(4) - don't mix (1,4) KP for calculating distance – Drawplate

Note for (5): You have to use the KP from the reference/ideal img for calculating the distances. kp_bef is refering to the kp in the real, raw scene, as certainly kp_aft are the stretched positions taken from the real image. I.e. compare the distance of the ideal kp with the kp from the real but stretched scene – Drawplate

Sorry for the late response, I have asked about this to my friends. I think I have totally misunderstood this question. What I was trying to do is: use the homography matrix to decide if the template image has been successfully found on the test image. I don't have any real point or any other information about the test image. So I think my answer here is irrelevant to the question asked, and also not a correct approach for classification. Thank you very much for your time and effort. – Oversubscribe

if I don't have any real point or any other information about the test image means you don't know what sign is to be expected then yes indeed you have to step up to machine learning for instance. Otherwise this approach still is valid (even though comparing the angles between the spots before/after might increase even more the robustness ;) – Drawplate

Yes, I was trying to use the matching results to predict which sign is present in the image. – Oversubscribe

If you have no more than n signs to distinguish (with no variation such as dirt, snow etc. just like your STOP-example above) then you're perfectly fine to establish an algorithm with keypoint matching. However I can't tell if the accuracy or processing time would be better if ML is used instead. – Drawplate

My target images are streetview captures so there may be many variations. I could able to detect the places of the signs (using shape and color based matching, mostly depend on cv.MatchShapes) however I am strugling on predicting which sign it is. I wonder if it is possible with single templates of each sign, I couln't find any yet. I will try SVM methods and prepare some training data. I wish I could have a face to face chat with you. – Oversubscribe

I'm not that familar with the properties of SVM's in such areas, mostly I've read about neuronal networks in that context. If you are still insecure what suits the best, to get a better overview have a look at face recognition papers, as it is pretty much the same situation but even more complicated (i.e. what is working there should greatly be a solution for you!). Btw I guess you should have opened up a new topic ;) – Drawplate