2D geometric shape vertices coordinates detection

Question

I am trying to find the vertices and their coordinates for simple geometric shape using marvin-framework.

This is the code I have (based on https://mcmap.net/q/708560/-2d-shape-recognition-algorithm-looking-for-guidance-closed)

package com.example.marvin;

import static marvin.MarvinPluginCollection.floodfillSegmentation;
import static marvin.MarvinPluginCollection.moravec;
import static marvin.MarvinPluginCollection.scale;

import java.io.FileWriter;
import java.io.IOException;

import marvin.image.MarvinImage;
import marvin.image.MarvinSegment;
import marvin.io.MarvinImageIO;

public class ShapesExample {

    private FileWriter fw = null;

    public ShapesExample() throws IOException{
        fw = new FileWriter("out.txt");

        // Scale down the image since the desired features can be extracted
        // in a lower resolution.
        MarvinImage image = MarvinImageIO.loadImage("square.png");
        scale(image.clone(), image, 269);

        // segment each object
        MarvinSegment[] objs = floodfillSegmentation(image);
        MarvinSegment seg;  

        OUT("Number of objects: " + objs.length);

        // For each object...
        // Skip position 0 which is just the background
        for(int i=1; i<objs.length; i++){
            seg = objs[i];
            OUT("seq: " + seg);
            MarvinImage imgSeg = image.subimage(seg.x1-5, seg.y1-5, seg.width+10, seg.height+10);
            OUT("i = " + i + "/" + objs.length);
            int[][] output;
            output = moravec(imgSeg, null, 18, 1000000);
            int xcount = 0;
            for(int x = 0; x < output.length; x++) {
                OUT("x = " + xcount++ + "/" + output[x].length);
                for(int y = 0; y < output[y].length; y++) {
                    OUTNONL("y = " + output[x][y] + " ");
                }
                OUT("");
            }
        }

        fw.close();
    }

    private void OUTNONL(String str) throws IOException {
        System.out.print(str);
    }

    private void OUT(String str) throws IOException {
        System.out.println(str);
    }

    public static void main(String[] args) throws IOException {
        new ShapesExample();
    }
}

square.png contains the shape image

This is the output I see:

Number of objects: 3
seq: {x1:97, x2:136, y1:35, y2:72, width:40, height:38, area:189}
i = 1/3
x = 0/48
y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 
.....
x = 48/48
y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 
seq: {x1:99, x2:135, y1:36, y2:71, width:37, height:36, area:1333}
i = 2/3
x = 0/46
y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 
.....
x = 46/46
y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 y = 0 

(where i is nth object detected in image, x is row, y is column) As you can see all values are 0.

Question:

• what am i missing in getting the coordinates?
• why does it detect 3 shapes in the image?

Hollinger

Answer 1

"why does it detect 3 shapes in the image?"

FloodfillSegmentation fill areas of pixels having distinct colors. In your case there are three separated areas: (1) white background, (2) square lines, (3) square interior. To solve this, I simply transformed your square in a solid shape:

 boundaryFill(image.clone(), image, 1, 1, Color.BLACK);
 invertColors(image);

"what am i missing in getting the coordinates?"

The moravec output is not a list of vertices. It's a map of pixels having an intensity value related to corners positions. I added getVertices(int[][] cornernessMap, int minDistanceBetweenPoints) method to your application in order to get the correct position of the vertices.

SOLUTION

output image:

output text:

Number of objects: 1
Vertex: (147,54)
Vertex: (147,105)
Vertex: (200,54)
Vertex: (200,105)

source code:

import static marvin.MarvinPluginCollection.boundaryFill;
import static marvin.MarvinPluginCollection.floodfillSegmentation;
import static marvin.MarvinPluginCollection.invertColors;
import static marvin.MarvinPluginCollection.moravec;
import static marvin.MarvinPluginCollection.scale;

import java.awt.Color;
import java.awt.Point;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

import marvin.image.MarvinImage;
import marvin.image.MarvinSegment;
import marvin.io.MarvinImageIO;

public class ShapesExample {

    private FileWriter fw = null;

    public ShapesExample() throws IOException{
        fw = new FileWriter("out.txt");

        // Scale down the image since the desired features can be extracted
        // in a lower resolution.
        MarvinImage image = MarvinImageIO.loadImage("./res/square.png");
        scale(image.clone(), image, 400);

        // Transform the square in a solid shape
        boundaryFill(image.clone(), image, 1, 1, Color.BLACK);
        invertColors(image);

        MarvinImageIO.saveImage(image, "./res/square_2.png");

        // segment each object
        MarvinSegment[] objs = floodfillSegmentation(image);
        MarvinSegment seg;  

        OUT("Number of objects: " + (objs.length-1));

        int MATRIX_SIZE = 5;
        // For each object...
        // Skip position 0 which is just the background
        for(int i=1; i<objs.length; i++){
            seg = objs[i];

            int[][] output = moravec(image, null, MATRIX_SIZE, 1000);
            List<Point> vertices = getVertices(output, 10);

            // Draw the vertices and print coordinates
            for(Point p:vertices) {
                image.fillRect(p.x-(MATRIX_SIZE/2), p.y-(MATRIX_SIZE/2), 5, 5, Color.red); 
                OUT("Vertex: ("+p.x+","+p.y+")");
             }
        }

        MarvinImageIO.saveImage(image, "./res/square_out.png");
        fw.close();
    }

    private List<Point> getVertices(int[][] cornernessMap, int minDistanceBetweenPoints){
        int corners=0;
        List<Point> points = new ArrayList<Point>();
        for(int x=0; x<cornernessMap.length; x++){
            for(int y=0; y<cornernessMap[0].length; y++){
                // Is it a corner?
                if(cornernessMap[x][y] > 0){
                    // This part of the algorithm avoid inexistent corners
                    // detected almost in the same position due to noise.
                    Point newPoint = new Point(x,y);
                    if(points.size() == 0){
                        points.add(newPoint); corners++;
                    }else {
                        boolean valid=true;
                        for(Point p:points){
                            if(newPoint.distance(p) < minDistanceBetweenPoints){
                                valid=false;
                            }
                        }
                        if(valid){
                            points.add(newPoint); corners++;
                        }
                    }
                }
            }
        }
        return points;
    }

    private void OUTNONL(String str) throws IOException {
        System.out.print(str);
    }

    private void OUT(String str) throws IOException {
        System.out.println(str);
    }

    public static void main(String[] args) throws IOException {
        new ShapesExample();
    }
}

Stucco

Hi Gabriel, thanks a lot! My goal is to given an input image, identify the shape. Given the vertices, I can determine the shapes through my own algorithm. However, the algorithm takes the inputs: number of vertices (v) and coordinates of vertices (c). For ex: v=4, c=0,0;0,10;10,0;10,10 is a square, v=3, c=0,0;0,10,10,0 is a triangle. etc. – Hollinger

(Continued) Two options: 1. Let marvin detect the shape directly from image (is there such a function?) 2. Marvin gives the output in above format. The current output has a lot of coordinates which are not necessarily vertices. Do you think i can get the output in required format? Which of the above two options you suggest? (I am accepting your answer since it answers my original question. This is an additional question - will post separately if you so desire) – Hollinger