Image Processing - Dress Segmentation using opencv

Question

I am working on dress feature identification using opencv. As a first step, I need to segment t-shirt by removing face and hands from the image. Any suggestion is appreciated.

Opec

start by looking at skin detection solutions (example: pyimagesearch.com/2014/08/18/…) – Bramlett

@RosaGronchi I would disagree with your edit. Because you have referred us to Adrian Rosebrock's skin detection algorithm to start, there is some pre-processing required before detecting the skin of an image and that requires image morphology and hence image processing. I have restored the tag as well as adding in image segmentation as we are technically segmenting out the skin as well. – Pasley

@Pasley segmentation is a classical computer vision problem, many times computer vision algorithms involve image processing tools but the question: I want to extract the shirt pixels out of an image" has nothing to do with image processing. You can train a DNN to do that as well (which won't make it a deep learning problem) – Bramlett

@RosaGronchi the point is that image processing is used to get to the final point of destination and so its use is merited. That's why I think the tag should be there. Either way, if we keep disagreeing, perhaps we can agree to disagree and say that's that. – Pasley

So this was considered a good question 2 years ago? If you were to post this question today it'd be closed pretty quick for being too broad, with comments like "post what you have tried so far" and "SO is not a code writing service". – Unparalleled

@CrisLuengo I think this is the definition of "Too Broad", but sometimes a question gets upvoted because it has a good answer – Olivia

Answer 1

I suggest the following approach:

1. Use Adrian Rosebrock's skin detection algorithm for detecting the skin (thank you for Rosa Gronchi for his comment).
2. Use region growing algorithm on the variance map. The initial seed can be calculated by using stage 1(see the attached code for more information).

code:

%stage 1: skin detection -  Adrian Rosebrock solution
im = imread(<path to input image>);
hsb = rgb2hsv(im)*255;

skinMask = hsb(:,:,1) > 0 & hsb(:,:,1) < 20;
skinMask = skinMask & (hsb(:,:,2) > 48 & hsb(:,:,2) < 255);
skinMask = skinMask & (hsb(:,:,3) > 80 & hsb(:,:,3) < 255);
skinMask = imclose(skinMask,strel('disk',6));

%stage 2: calculate top, left and right centroid from the different connected
%components of the skin
stats = regionprops(skinMask,'centroid');
topCentroid = stats(1).Centroid;
rightCentroid = stats(1).Centroid;
leftCentroid = stats(1).Centroid;
for x = 1 : length(stats)
    centroid = stats(x).Centroid;
    if topCentroid(2)>centroid(2)
        topCentroid = centroid;
    elseif centroid(1)<leftCentroid(1)
        leftCentroid = centroid;
    elseif centroid(1)>rightCentroid(1)
        rightCentroid = centroid;
    end
end

%first seed - the average of the most left and right centroids.
centralSeed = int16((rightCentroid+leftCentroid)/2);

%second seed - a pixel which is right below the face centroid.
faceSeed = int16(topCentroid);
faceSeed(2) = faceSeed(2)+40; 

%stage 3: std filter
varIm = stdfilt(rgb2gray(im));

%stage 4 - region growing on varIm  using faceSeed and centralSeed
res1=regiongrowing(varIm,centralSeed(2),centralSeed(1),8);
res2=regiongrowing(varIm,faceSeed(2),faceSeed(1),8);
res = res1|res2;

%noise reduction
res = imclose(res,strel('disk',3));
res = imopen(res,strel('disk',2));

result after stage 1(skin detection):

final result:

Comments:

1. Stage 1 is calculated using the following algorithm.
2. The region growing function can be downloaded here.
3. The solution is not perfect. For example, it may fail if the texture of the shirt is similar to the texture of the background. But I think that it can be a good start.
4. Another improvement which can be done is to use a better region growing algorithm, which doesn't grows into the skinMask location. Also, instead of using the region growing algorithm twice independently, the result of the second call of region growing can can be based on the result from the first one.

Rios