Segment pixels in an image based on colour (Matlab)

Question

I'm trying to segment an image containing multiple lego bricks using colour information only (for now). The aim is to find lego bricks that e.g. are green. I have tried using k-means clustering, but the number of different coloured bricks present in a given varies. I have also tried using the following example from the Matlab website

but that wasn't successful. Is there a simple way of segmenting based on colour?

An example image for the problem:

Tubular

Since the number of lego bricks having different colors vary, this problem becomes a little difficult. You can try graph-based image segmentation from here or here. You may also consider doing some pre-processing before assigning initial centroids to k-means. – Dodie

Similar problem as this question. You may find some useful tidbits there too. – Bifocals

Answer 1

So RGB or LAB colorspaces aren't really the best ones to use when choosing regions based on color alone. The better choice is HSV (Hue-Saturation-Value). Here we can define what hue ranges define 'green', a parameter for the saturation to defined what is a 'colored' pixel, and a minimum region size. Then some thresholding based on those values, some morphological filtering, and filtering of the regions that are returned before plotting. The usual routine.

The code below detects the green bricks in your provided image. It's not quite perfect because adjacent bricks are returned as a single region, but you can do some more exhaustive work inside these detected regions with an edge filter, for example, to end up with a precise count of the number of bricks.

% Input image
img = imread('https://i.sstatic.net/HSYc1.jpg');

greenRange = [0.4 0.5]; % Range of hue values considered 'green'
minSat = 0.5; % Minimum saturation value for 'colored' pixels to exclude bkgd noise
minRegionsize = 500; % Min size for a single block

%%%%%%%%%%%%%%%%%%%
% Denoise with a gaussian blur
imgfilt = imfilter(img, fspecial('gaussian', 10, 2));
% Convert image to HSV format
hsvImg = rgb2hsv(imgfilt);

% Threshold hue to get only green pixels and saturation for only colored
% pixels
greenBin = hsvImg(:,:,1) > greenRange(1) & hsvImg(:,:,1) < greenRange(2) & hsvImg(:,:,2) > minSat;
greenBin = bwmorph(greenBin, 'close'); % Morphological closing to take care of some of the noisy thresholding

% Use regionprops to filter based on area, return location of green blocks
regs = regionprops(greenBin, 'Area', 'Centroid', 'BoundingBox');
% Remove every region smaller than minRegionSize
regs(vertcat(regs.Area) < minRegionsize) = [];

% Display image with bounding boxes overlaid
figure()
image(img);
axis image
hold on
for k = 1:length(regs)
    plot(regs(k).Centroid(1), regs(k).Centroid(2), 'cx');

    boundBox = repmat(regs(k).BoundingBox(1:2), 5, 1) + ...
        [0 0; ...
        regs(k).BoundingBox(3) 0;...
        regs(k).BoundingBox(3) regs(k).BoundingBox(4);...
        0 regs(k).BoundingBox(4);...
        0 0];    
    plot(boundBox(:,1), boundBox(:,2), 'r');
end
hold off

Selfinduced

Second question is an easier one. For hue ranges take a look at a color selection tool that includes HSV values like this one: hslpicker.com/#00ff6f . You need the hue value between 0 and 1 not 0 and 255 for MATLAB though. Or look at the values in the first slice of the hsvImg matrix above. – Selfinduced

I sort of played around with the values to see what colours they correspond to, but that tool looks quite useful! Any ideas on the first question? – Tubular

Red will have values around 0, Blue around 0.33, Green around 0.67, but these are for R, G, or B values. The 'green' bricks are more cyan than pure green, so the hue value for those is skewed towards blue. But really I looked at the values in the first slice of hsvImg and estimated. – Selfinduced

Using the min region size will remove the smaller regions detected, but that doesnt necessarily correspond to the larger bricks. The bricks are not cleanly thresholded since the lighting is very uneven. Look at greenBin above and see how messy it is. Playing with the hue and saturation threshold values and the blur filter might help with that, but I dont expect it to be perfect ever. – Selfinduced

You can instead use an edge filter to pick out the edges in each region corresponding to a green brick and figure out the number of holes that way (or the size from the bordering edges). That is a more complicated problem than picking out the green bricks without a cleaner picture. A cleaner one would be with more even lighting and at a more downward angle. – Selfinduced

Good job in using HSV. It's the ideal colour space for doing colour segmentation. – Dardan

Suggestion for you. You can directly use the BoundingBox attribute into the rectangle function, which accepts the four element array directly as an input. No need to replicate the points along each side to draw the rectangle. Using rectangle makes drawing rectangles much easier. – Dardan