raise ValueError('Image with id {} already added.'.format(image_id)) in Tensorflow object detection api

Question

Image training is ok with ssd_mobilenet_v1_coco in tensorflow object detection api.

getting the error while testing:

File "/home/hipstudents/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/object_detection-0.1-py3.6.egg/object_detection/utils/object_detection_evaluation.py", line 203, in add_single_ground_truth_image_info
raise ValueError('Image with id {} already added.'.format(image_id))

Please help.

System Info:

What is the top-level directory of the model you are using: ~/
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes, written scripts to convert .xml files to tf record 
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
TensorFlow installed from (source or binary): Compiled from source
TensorFlow version (use command below): 1.11.0
Bazel version (if compiling from source): 0.16.1
CUDA/cuDNN version: 9.0.176, cuDNN: 9.0
GPU model and memory: GeForce GTX1080Ti, 11GB
Exact command to reproduce: python eval.py --logtostderr --pipeline_config_path=training/ssd_mobilenet_v1_coco.config --checkpoint_dir=training/ --eval_dir=eval/

I created dataset manually. Then label it using labelimg. after labeling I created csv file for image annotation and file name. then I create tf record. I follow this tutorial: https://towardsdatascience.com/how-to-train-your-own-object-detector-with-tensorflows-object-detector-api-bec72ecfe1d9

My tfrecord generator for training and testing image:

"""
Usage:
  # From tensorflow/models/
  # Create train data:
  python generate_tfrecord.py --csv_input=data/train_labels.csv  --output_path=train.record
  # Create test data:
  python generate_tfrecord.py --csv_input=data/test_labels.csv  --output_path=test.record
"""
from __future__ import division
from __future__ import print_function
from __future__ import absolute_import

import os
import io
import pandas as pd
import tensorflow as tf

from PIL import Image
from object_detection.utils import dataset_util
from collections import namedtuple, OrderedDict

flags = tf.app.flags
flags.DEFINE_string('csv_input', '', 'Path to the CSV input')
flags.DEFINE_string('output_path', '', 'Path to output TFRecord')
FLAGS = flags.FLAGS


# TO-DO replace this with label map
def class_text_to_int(row_label):
    if row_label == 'Field':
        return 1
    else:
        None


def split(df, group):
    data = namedtuple('data', ['filename', 'object'])
    gb = df.groupby(group)
    return [data(filename, gb.get_group(x)) for filename, x in zip(gb.groups.keys(), gb.groups)]


def create_tf_example(group, path):
    with tf.gfile.GFile(os.path.join(path, '{}'.format(group.filename)), 'rb') as fid:
        encoded_jpg = fid.read()
    encoded_jpg_io = io.BytesIO(encoded_jpg)
    image = Image.open(encoded_jpg_io)
    width, height = image.size

    filename = group.filename.encode('utf8')
    image_format = b'jpg'
    xmins = []
    xmaxs = []
    ymins = []
    ymaxs = []
    classes_text = []
    classes = []

    for index, row in group.object.iterrows():
        xmins.append(row['xmin'] / width)
        xmaxs.append(row['xmax'] / width)
        ymins.append(row['ymin'] / height)
        ymaxs.append(row['ymax'] / height)
        classes_text.append(row['class'].encode('utf8'))
        classes.append(class_text_to_int(row['class']))

    tf_example = tf.train.Example(features=tf.train.Features(feature={
        'image/height': dataset_util.int64_feature(height),
        'image/width': dataset_util.int64_feature(width),
        'image/filename': dataset_util.bytes_feature(filename),
        'image/source_id': dataset_util.bytes_feature(filename),
        'image/encoded': dataset_util.bytes_feature(encoded_jpg),
        'image/format': dataset_util.bytes_feature(image_format),
        'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
        'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
        'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
        'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
        'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
        'image/object/class/label': dataset_util.int64_list_feature(classes),
    }))
    return tf_example


def main(_):
    writer = tf.python_io.TFRecordWriter(FLAGS.output_path)
    path = os.path.join(os.getcwd(), 'Images')
    examples = pd.read_csv(FLAGS.csv_input)
    grouped = split(examples, 'filename')
    for group in grouped:
        tf_example = create_tf_example(group, path)
        writer.write(tf_example.SerializeToString())

    writer.close()
    output_path = os.path.join(os.getcwd(), FLAGS.output_path)
    print('Successfully created the TFRecords: {}'.format(output_path))


if __name__ == '__main__':
    tf.app.run()

Foreskin

Answer 1

In the ssd_mobilenet_coco_v1.config file, num_examples was 8000. In my case, test dataset only has 121 samples. I forgot to update that and got new kind of error that I couldn't find on the Internet. As it is a silly mistake, so I think a very few people did that. this answer might help someone who will do this kind of mistake. I changed the following in the config file and the error is resolved:

eval_config: {
        #num of test images. In my case 121. Previously It was 8000
        num_examples: 121
        # Note: The below line limits the evaluation process to 10 evaluations.
        # Remove the below line to evaluate indefinitely.
        max_evals: 10
      }

Foreskin

Answer 2

In my case, the problem was that I'd included images multiple times when constructing tfrecord files. Though now obvious, I hadn't noticed that many categories of the Open Images Dataset share the same images (which would have the same id in the evaluation, thus the error...). Once I'd corrected the algorithm creating the tfrecords, the error was gone.

Infest

Answer 3

I have solved the problem with this article: https://www.coder.work/article/3120495

By just adding 2 line

eval_config {
  num_examples: 50
  use_moving_averages: false
  metrics_set: "coco_detection_metrics"
}

Roughrider