How to train Tensorflow Object Detection images that do not contain objects?

Question

I am training an object detection network using Tensorflow's object detection,

https://github.com/tensorflow/models/tree/master/research/object_detection

I can successfully train a network based on my own images and labels. However, I have a large dataset of images that do not contain any of my labeled objects, and I want to be able to train the network to not detect anything in these images.

From what I understand with Tensorflow object detection, I need to give it a set of images and corresponding XML files that box and label the objects in the image. The scripts convert the XML to CSV and then to another format for the training, and do not allow XML files that have no objects.

How to give an image and XML files that have no objects?

Or, how does the network learn what is not an object?

For example if you want to detect "hot dogs" you can train it with a set of images with hot dogs. But how to train it what is not a hot dog?

Foreshow

Seems like this is an issue with the scripts we are using from github.com/datitran/raccoon_dataset they convert the ImageNet XML files to csv and then to tf-records, I will see if I can fix the scripts, assuming tf-records can train with no objects? – Foreshow

We ended up fixing the script to support empty xml file when converting to csv and tfrecords. We also created a script that generates an empty xml file for an image set (of no objects) – Foreshow

Can you share us how to fix this issue? – Kopaz

Answer 1

An Object Detection CNN can learn what is not an object, simply by letting it see examples of images without any labels.

There are two main architecture types:

1. two-stages, with first stage object/region proposal (RPN), and second - classification and bounding box fine-tuning;
2. one-stage, which directly classifies and regresses BB based on the feature vector corresponding to a certain cell in the feature map.

In any case, there's a part which is responsible to decide what is an object and what's not. In RPN you have "objectness" score, and in one-stages there's the confidence of classification, where you usually a background class (i.e. everything which is not the supported classes).

So in both cases, in case a specific example in an image doesn't have any supported class, you teach the CNN to decrease the objectness score or increase the background confidence correspondingly.

Brittenybrittingham

Thanks for the info. The python scripts from github.com/tensorflow/models/tree/master/research/… do not allow for an image's XML file to have no objects, so must be just a script issue. – Foreshow

Answer 2

You might want to take a look at this solution.

For for the tensorflow object detection API to include your negative examples, you need to add the negative examples to the csv file you have created from the xml, either by modifying the script that generates the csv file or by adding the examples afterwards.

For generating xml-files without class labels using LabelImg, you can do this by pressing "Verify Image".

Picofarad