Tensorflow/models uses COCO 90 class ids although COCO has only 80 categories
Asked Answered
U

3

12

The labelmaps of Tensorflows object_detection project contain 90 classes, although COCO has only 80 categories. Therefore the parameter num_classes in all sample configs is set to 90.

If i now download and use the COCO 2017 dataset, do I need to set this parameter to 80 or leave it to 90?

If 80 (as COCO has 80 classes) I need to adjust the labelmap, so the standard mscoco_label_map.pbtxt is not correct, right?

I would be really thankful if someone could shine a light on this one :)

Here are the standard 80 COCO classes:

person
bicycle
car
motorbike
aeroplane
bus
train
truck
boat
traffic light
fire hydrant
stop sign
parking meter
bench
bird
cat
dog
horse
sheep
cow
elephant
bear
zebra
giraffe
backpack
umbrella
handbag
tie
suitcase
frisbee
skis
snowboard
sports ball
kite
baseball bat
baseball glove
skateboard
surfboard
tennis racket
bottle
wine glass
cup
fork
knife
spoon
bowl
banana
apple
sandwich
orange
broccoli
carrot
hot dog
pizza
donut
cake
chair
sofa
pottedplant
bed
diningtable
toilet
tvmonitor
laptop
mouse
remote
keyboard
cell phone
microwave
oven
toaster
sink
refrigerator
book
clock
vase
scissors
teddy bear
hair drier
toothbrush

And here is the MS COCO labelmap of Tensorflows object_detection API:

item {
  name: "/m/01g317"
  id: 1
  display_name: "person"
}
item {
  name: "/m/0199g"
  id: 2
  display_name: "bicycle"
}
item {
  name: "/m/0k4j"
  id: 3
  display_name: "car"
}
item {
  name: "/m/04_sv"
  id: 4
  display_name: "motorcycle"
}
item {
  name: "/m/05czz6l"
  id: 5
  display_name: "airplane"
}
item {
  name: "/m/01bjv"
  id: 6
  display_name: "bus"
}
item {
  name: "/m/07jdr"
  id: 7
  display_name: "train"
}
item {
  name: "/m/07r04"
  id: 8
  display_name: "truck"
}
item {
  name: "/m/019jd"
  id: 9
  display_name: "boat"
}
item {
  name: "/m/015qff"
  id: 10
  display_name: "traffic light"
}
item {
  name: "/m/01pns0"
  id: 11
  display_name: "fire hydrant"
}
item {
  name: "/m/02pv19"
  id: 13
  display_name: "stop sign"
}
item {
  name: "/m/015qbp"
  id: 14
  display_name: "parking meter"
}
item {
  name: "/m/0cvnqh"
  id: 15
  display_name: "bench"
}
item {
  name: "/m/015p6"
  id: 16
  display_name: "bird"
}
item {
  name: "/m/01yrx"
  id: 17
  display_name: "cat"
}
item {
  name: "/m/0bt9lr"
  id: 18
  display_name: "dog"
}
item {
  name: "/m/03k3r"
  id: 19
  display_name: "horse"
}
item {
  name: "/m/07bgp"
  id: 20
  display_name: "sheep"
}
item {
  name: "/m/01xq0k1"
  id: 21
  display_name: "cow"
}
item {
  name: "/m/0bwd_0j"
  id: 22
  display_name: "elephant"
}
item {
  name: "/m/01dws"
  id: 23
  display_name: "bear"
}
item {
  name: "/m/0898b"
  id: 24
  display_name: "zebra"
}
item {
  name: "/m/03bk1"
  id: 25
  display_name: "giraffe"
}
item {
  name: "/m/01940j"
  id: 27
  display_name: "backpack"
}
item {
  name: "/m/0hnnb"
  id: 28
  display_name: "umbrella"
}
item {
  name: "/m/080hkjn"
  id: 31
  display_name: "handbag"
}
item {
  name: "/m/01rkbr"
  id: 32
  display_name: "tie"
}
item {
  name: "/m/01s55n"
  id: 33
  display_name: "suitcase"
}
item {
  name: "/m/02wmf"
  id: 34
  display_name: "frisbee"
}
item {
  name: "/m/071p9"
  id: 35
  display_name: "skis"
}
item {
  name: "/m/06__v"
  id: 36
  display_name: "snowboard"
}
item {
  name: "/m/018xm"
  id: 37
  display_name: "sports ball"
}
item {
  name: "/m/02zt3"
  id: 38
  display_name: "kite"
}
item {
  name: "/m/03g8mr"
  id: 39
  display_name: "baseball bat"
}
item {
  name: "/m/03grzl"
  id: 40
  display_name: "baseball glove"
}
item {
  name: "/m/06_fw"
  id: 41
  display_name: "skateboard"
}
item {
  name: "/m/019w40"
  id: 42
  display_name: "surfboard"
}
item {
  name: "/m/0dv9c"
  id: 43
  display_name: "tennis racket"
}
item {
  name: "/m/04dr76w"
  id: 44
  display_name: "bottle"
}
item {
  name: "/m/09tvcd"
  id: 46
  display_name: "wine glass"
}
item {
  name: "/m/08gqpm"
  id: 47
  display_name: "cup"
}
item {
  name: "/m/0dt3t"
  id: 48
  display_name: "fork"
}
item {
  name: "/m/04ctx"
  id: 49
  display_name: "knife"
}
item {
  name: "/m/0cmx8"
  id: 50
  display_name: "spoon"
}
item {
  name: "/m/04kkgm"
  id: 51
  display_name: "bowl"
}
item {
  name: "/m/09qck"
  id: 52
  display_name: "banana"
}
item {
  name: "/m/014j1m"
  id: 53
  display_name: "apple"
}
item {
  name: "/m/0l515"
  id: 54
  display_name: "sandwich"
}
item {
  name: "/m/0cyhj_"
  id: 55
  display_name: "orange"
}
item {
  name: "/m/0hkxq"
  id: 56
  display_name: "broccoli"
}
item {
  name: "/m/0fj52s"
  id: 57
  display_name: "carrot"
}
item {
  name: "/m/01b9xk"
  id: 58
  display_name: "hot dog"
}
item {
  name: "/m/0663v"
  id: 59
  display_name: "pizza"
}
item {
  name: "/m/0jy4k"
  id: 60
  display_name: "donut"
}
item {
  name: "/m/0fszt"
  id: 61
  display_name: "cake"
}
item {
  name: "/m/01mzpv"
  id: 62
  display_name: "chair"
}
item {
  name: "/m/02crq1"
  id: 63
  display_name: "couch"
}
item {
  name: "/m/03fp41"
  id: 64
  display_name: "potted plant"
}
item {
  name: "/m/03ssj5"
  id: 65
  display_name: "bed"
}
item {
  name: "/m/04bcr3"
  id: 67
  display_name: "dining table"
}
item {
  name: "/m/09g1w"
  id: 70
  display_name: "toilet"
}
item {
  name: "/m/07c52"
  id: 72
  display_name: "tv"
}
item {
  name: "/m/01c648"
  id: 73
  display_name: "laptop"
}
item {
  name: "/m/020lf"
  id: 74
  display_name: "mouse"
}
item {
  name: "/m/0qjjc"
  id: 75
  display_name: "remote"
}
item {
  name: "/m/01m2v"
  id: 76
  display_name: "keyboard"
}
item {
  name: "/m/050k8"
  id: 77
  display_name: "cell phone"
}
item {
  name: "/m/0fx9l"
  id: 78
  display_name: "microwave"
}
item {
  name: "/m/029bxz"
  id: 79
  display_name: "oven"
}
item {
  name: "/m/01k6s3"
  id: 80
  display_name: "toaster"
}
item {
  name: "/m/0130jx"
  id: 81
  display_name: "sink"
}
item {
  name: "/m/040b_t"
  id: 82
  display_name: "refrigerator"
}
item {
  name: "/m/0bt_c3"
  id: 84
  display_name: "book"
}
item {
  name: "/m/01x3z"
  id: 85
  display_name: "clock"
}
item {
  name: "/m/02s195"
  id: 86
  display_name: "vase"
}
item {
  name: "/m/01lsmm"
  id: 87
  display_name: "scissors"
}
item {
  name: "/m/0kmg4"
  id: 88
  display_name: "teddy bear"
}
item {
  name: "/m/03wvsk"
  id: 89
  display_name: "hair drier"
}
item {
  name: "/m/012xff"
  id: 90
  display_name: "toothbrush"
}

Edit: after closely comparing the two lists it is clear that they both contain the same 80 classes but the label map tensorflow uses by default misses 10 class ids, seemingly random distributed.

Has anybody an idea why that is?

Untimely answered 3/6, 2018 at 9:50 Comment(0)
B
9

The MSCOCO paper describes that the dataset has actually 91 classes but in the 2014 dataset they released only a subset of 80 classes because they didn't annotated the segmentation of the remaining 11 classes. Seems that tensorflow models were trained using 90 classes.

MSCOCO paper: https://arxiv.org/pdf/1405.0312.pdf

From appendix II: "Our dataset contains 91 object categories (the 2014 release contains segmentation masks for 80 of these categories)."

-Ricardo

Bytom answered 17/11, 2018 at 14:39 Comment(1)
Are 91 objects 90+1, I.e. does this list include background class?Live
M
1

I added in placeholders where there were missing classes in the label map. This can be used when running inference or when adding metadata to the model.

person
bicycle
car
motorcycle
airplan
bus
train
truck
boat
traffic light
fire hydrant
placeholder1
stop sign
parking meter
bench
bird
cat
dog
horse
sheep
cow
elephant
bear
zebra
giraffe
placeholder2
backpack
umbrella
placeholder3
placeholder4
handbag
tie
suitcase
frisbee
skis
snowboard
sports ball
kite
baseball bat
baseball glove
skateboard
surfboard
tennis racket
bottle
placeholder5
wine glass
cup
fork
knife
spoon
bowl
banana
apple
sandwich
orange
broccoli
carrot
hot dog
pizza
donut
cake
chair
couch
potted plant
bed
placeholder6
dining table
placeholder7
placeholder8
toilet
placeholder9
tv
laptop
mouse
remote
keyboard
cell phone
microwave
oven
toaster
sink
refrigerator
placeholder10
book
clock
vase
scissors
teddy bear
hair drier
toothbrush
Mog answered 23/8, 2023 at 14:25 Comment(0)
M
0

You don't need to change 80 to 90, I think the num_classes in the config is just the max id of object classes. More refer to https://github.com/tensorflow/models/issues/1719

Metrical answered 14/3, 2019 at 8:57 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.