Mask for BackgroundSubtractorMOG2

Question

How can I tell BackgroundSubtractorMOG2 which pixels to update to the background model and which pixels shouldn't.

I am facing problem when there's an object entered the scene and stopped for a few ten seconds, the object will be absorbed into the background model.

I wanted to decrease the learning rate or stop the learning around the particular stopped object but how can I do that? Does BackgroundSubtractorMOG2 support using mask in its update function?

I am using OpenCV 2.4.1.

Ignite

Good question... I am facing the exact same problem. – Enlace

There is no mechanism for high level feedback to OpenCV background subtraction algorithms. My suggestion for a somewhat generic update function would include an update mask. – Semirigid

Answer 1

BackgroundSubtractorMOG2 does not support masking the input. But, if you know which pixels you want to mask you can mask the output: say you've called subtractor(input, fg, learningRate); and you somehow know where the object is now (may be you've been tracking it using mean shift or pattern recognition) just do fg |= mask; where mask is where, as you know from some different source, the object is.

Strathspey

that does help in tracking foreground that's being absorbed into the background. but this is not the answer i'm seeking for, i would like to keep my background model clean if possible. – Ignite

Even my first sentence isn't the answer you seek? If your objects are such that they come in and stay still, the MOG2 model isn't enough to track them. you'd be better off combining MOG2 for initial object detection and some object tracking algorithm for tracking it afterwards. Or do I misunderstand your problem? – Strathspey

Answer 2

You can accomplish this by setting the learning rate down really low

ie:

mog(input, output, 0.00000001);

Conley

Answer 3

you can replace masked parts with background image:

BackgroundSubtractorMOG2 mog_bgs;
.
.
void my_apply(const cv::Mat& img, cv::Mat& fg, const cv::Mat& mask){
  cv::Mat last_bg;
  cv::Mat masked_img = img.clone();
  mog_bgs.getBackgroundImage(last_bg);
  last_bg.copyTo(masked_img, mask);
  mog_bgs.apply(masked_img, fg);
}

or weight masked parts:

BackgroundSubtractorMOG2 mog_bgs;
.
.
void my_apply(const cv::Mat& img, cv::Mat& fg, const cv::Mat& mask){
  cv::Mat last_bg;
  cv::Mat masked_img = img.clone();
  mog_bgs.getBackgroundImage(last_bg);
  masked_img.forEach<Vec3b>([&](Vec3b& p, const int* position) -> void
    {
        if (mask.at<uchar>(position[0], position[1]) > 0) {
            auto b = last_bg.at<Vec3b>(position[0], position[1]);
            p[0] = p[0]*0.2 + b[0]*0.8;
            p[1] = p[1]*0.2 + b[1]*0.8;
            p[2] = p[2]*0.2 + b[2]*0.8;
        }
    });
  mog_bgs.apply(masked_img, fg);
}

Tableau