Get the average color inside a contour with Open CV

Question

So I decided to get started learning Open CV and Python together!

My first project is to detect moving objects on a relatively still background and then detect their average color to sort them. There are at least 10 objects to detect and I am processing a colored video.

So far I managed to remove the background, identify the contours (optionally get the center of each contour) but now I am struggling getting the average or mean color inside of each contour. There are some topics about this kind of question but most of them are written in C. Apparently I could use cv.mean() but I can't get a working mask to feed in this function. I guess it's not so difficult but I am stuck there... Cheers!

import numpy as np
import cv2

video_path = 'test.h264'

cap = cv2.VideoCapture(video_path)
fgbg = cv2.createBackgroundSubtractorMOG2()


while (cap.isOpened):

    ret, frame = cap.read()
    if ret==True:
        fgmask = fgbg.apply(frame)
        (contours, hierarchy) = cv2.findContours(fgmask, cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)

        for c in contours:
            if cv2.contourArea(c) > 2000:
                cv2.drawContours(frame, c, -1, (255,0,0), 3)
        cv2.imshow('foreground and background',fgmask)
        cv2.imshow('rgb',frame)

    key = cv2.waitKey(1) & 0xFF

    if key == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

Seagrave

"...detect their average color to sort them. There are at least 10 objects to detect and I am processing a colored video" - Please elaborate more on this. What you mean by that? How do you want to "sort" them? – Groundage

I want to indetify if the objects are mainly red, blue, black... any color actually. This could be like a M&M color sorting machine youtube.com/watch?v=H7HTQai7Wwg The difference is that the objects I am trying to sort are not so intensely colored as M&M but the program should be able to output the average color of each contour. Then can I program the GPIO (of my raspberry pi) accordingly. Do you see what I mean? – Seagrave

I am working on a color sorting machine. Each contour is an object to sort. My goal is to identify every white object (or any other color) and command the GPIO (of my raspberry pi) to remove them from the flow using a robotic arm for example. – Seagrave

Answer 1

You can create a mask by first creating a new image with the same dimensions as your input image and pixel values set to zero.

You then draw the contour(s) onto this image with pixel value 255. The resulting image can be used as a mask.

mask = np.zeros(frame.shape, np.uint8)
cv2.drawContours(mask, c, -1, 255, -1)

mask can then be used as a parameter to cv.mean like

mean = cv.mean(frame, mask=mask)

Just one word of caution, the mean of RGB colors does not always make sense. Maybe try converting to HSV color space and solely use the H channel for detecting the color of your objects.

Bancroft

Answer 2

Solution on an image

1) find contour (in this case rectangle, contour that is not rectangle is much harder to make)

2) find coordiantes of contour

3) cut the image from contour

4) sum individual channels and divide them by number of pixels in it ( or with mean function)

import numpy as np
import cv2
img = cv2.imread('my_image.jpg',1)
cp = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(cp,150,255,0)
cv2.imshow('img',thresh) 
cv2.waitKey(0)
im2,contours,hierarchy = cv2.findContours(thresh.astype(np.uint8), 1, 2)
cnts = contours
for cnt in cnts:
    if cv2.contourArea(cnt) >800: # filter small contours
        x,y,w,h = cv2.boundingRect(cnt) # offsets - with this you get 'mask'
        cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
        cv2.imshow('cutted contour',img[y:y+h,x:x+w])
        print('Average color (BGR): ',np.array(cv2.mean(img[y:y+h,x:x+w])).astype(np.uint8))
        cv2.waitKey(0)
cv2.imshow('img',img) 
cv2.waitKey(0)
cv2.destroyAllWindows()

To remove noise, you can just take center of the contour, and take smaller rectangle to examin.

For non rectangular contour, look at cv2.fillPoly function -> Cropping non rectangular contours. But its a bit slow algorithm (but nothing limiting)

If you are interested in non rectangular contour, you will have to be careful about doing mean, because you will need mask and the mask/background is always rectangular so you will be doing mean on something you dont want

Mcnally