color replacement in image for iphone application

Question

Basically i want to implement color replacement feature for my paint application. Below are original and expected output

Original:

After changing wall color selected by user along with some threshold for replacement

I have tried two approaches but could not got working as expected

Approach 1:
Queue-based Flood Fill algorithm for color replacement but with i got below output with terribly slow and wall shadow has not been preserved.

Approach 2: So i have tried to look at another option and found below post from SO How to change a particular color in an image?

but i could not understand logic and not sure about my code implementation from step 3.

Please find below code for each step wise with my understanding.

1) Convert the image from RGB to HSV using cvCvtColor (we only want to change the hue).

 IplImage *mainImage=[self CreateIplImageFromUIImage:[UIImage imageNamed:@"original.jpg"]];
 IplImage *hsvImage = cvCreateImage(cvGetSize(mainImage), IPL_DEPTH_8U, 3);
 IplImage *threshImage = cvCreateImage(cvGetSize(mainImage), IPL_DEPTH_8U, 3);
 cvCvtColor(mainImage,hsvImage,CV_RGB2HSV);

2) Isolate a color with cvThreshold specifying a certain tolerance (you want a range of colors, not one flat color).

cvThreshold(hsvImage, threshImage, 0, 100, CV_THRESH_BINARY);

3) Discard areas of color below a minimum size using a blob detection library like cvBlobsLib. This will get rid of dots of the similar color in the scene. Do i need to specify original image or thresold image?

CBlobResult blobs = CBlobResult(threshImage, NULL, 0);
blobs.Filter( blobs, B_EXCLUDE, CBlobGetArea(), B_LESS, 10);

4) Mask the color with cvInRangeS and use the resulting mask to apply the new hue.

Not sure about this function how it helps in color replacement and not able to understand arguments to be provided.

5) cvMerge the new image with the new hue with an image composed by the saturation and brightness channels that you saved in step one.

i understand that cvMerge will merge three channel of H S and V but how i can use output of above three steps.

so basically stuck with opencv implementation,

if possible then please guide me for opencv implemenation or any other solution to tryout.

Peridotite

It won't be enough to modify just the hue in step 4 and then use the original saturation in step 5. Look at the wall in your example. It's gray. Gray's saturation is zero (or very close to zero), meaning that the hue has no effect. To turn that wall blue, you need to set the saturation to something closer to one (and you need to set the hue). – Dormitory

rob do you think that my 2nd and 3rd step are correct? – Peridotite

@JigarParekh i have same problem but i have to do in real time(means in camera image) so can u guide how to do that – Vane

Answer 1

Finally i am able to achieve some desired output using below javacv code and same ported to opencv too.

this solution has 2 problems

1. don't have edge detection, i think using contours i can achieve it

2. replaced color has flat hue and sat which should set based on source pixel hue sat difference but not sure how to achieve that. may be instead of cvSet using cvAddS

   IplImage image = cvLoadImage("sample.png");
   CvSize cvSize = cvGetSize(image);
   
   
   IplImage hsvImage = cvCreateImage(cvSize, image.depth(),image.nChannels());
   
   IplImage hChannel = cvCreateImage(cvSize, image.depth(), 1); 
           IplImage  sChannel = cvCreateImage(cvSize, image.depth(), 1); 
           IplImage  vChannel = cvCreateImage(cvSize, image.depth(), 1);
   cvSplit(hsvImage, hChannel, sChannel, vChannel, null);
   
   
   IplImage cvInRange = cvCreateImage(cvSize, image.depth(), 1);
   CvScalar source=new CvScalar(72/2,0.07*255,66,0); //source color to replace
   CvScalar from=getScaler(source,false);
   CvScalar to=getScaler(source, true);
   
   cvInRangeS(hsvImage, from , to, cvInRange);
   
   IplImage dest = cvCreateImage(cvSize, image.depth(), image.nChannels());
   
   IplImage temp = cvCreateImage(cvSize, IPL_DEPTH_8U, 2);
   cvMerge(hChannel, sChannel, null, null, temp);
   
   cvSet(temp, new CvScalar(45,255,0,0), cvInRange);// destination hue and sat
   cvSplit(temp, hChannel, sChannel, null, null);
   cvMerge(hChannel, sChannel, vChannel, null, dest);
   cvCvtColor(dest, dest, CV_HSV2BGR);
   cvSaveImage("output.png", dest);
   

method to for calculating threshold

    CvScalar getScaler(CvScalar seed,boolean plus){
    if(plus){
        return CV_RGB(seed.red()+(seed.red()*thresold),seed.green()+(seed.green()*thresold),seed.blue()+(seed.blue()*thresold));
    }else{
        return CV_RGB(seed.red()-(seed.red()*thresold),seed.green()-(seed.green()*thresold),seed.blue()-(seed.blue()*thresold));
    }
        }

Peridotite

Can you please share what material or tutorial you are referring for this feature ? I am also working on similar functionality .. can't find anything proper :( – Keep

Would be awesome if you can share – Mexico

Answer 2

I know this answer will be useful to someone someday. try this out in your view viewdidLoad() override method for iOS. image in the code snippet below should be from your UIImageView seed also is fixed.you can make it dynamic based on user tap event.

 cv::Mat mask = cv::Mat::zeros(image.rows + 2, image.cols + 2, CV_8U);
 imageView.image = [self UIImageFromCVMat:image];
 cv::cvtColor(image, image, cv::COLOR_BGR2RGB);

  try {
    if(seed.x > 0 && seed.y > 0){
        cv::floodFill(image, mask, seed, cv::Scalar(50, 155, 20) ,0,        cv::Scalar(2,2, 2), cv::Scalar(2,2, 2), 8);
        cv::floodFill(image, mask, seed2, cv::Scalar(50, 155, 20) ,0, cv::Scalar(2,2, 2), cv::Scalar(2,2, 2), 8);
        cv::floodFill(image, mask, seed3, cv::Scalar(50, 155, 0) ,0, cv::Scalar(2,2, 2), cv::Scalar(2,2, 2), 8);

    }
} catch (Exception ex) {

}
cv::cvtColor(image, image, cv::COLOR_RGB2BGR);

self.imageView.contentMode = UIViewContentModeScaleAspectFill;
self.imageView.image = [self UIImageFromCVMat:image];

Majewski