Nothing is being detected in Tensorflow Object detection API

Question

I'm trying to implement Tensorflow object detection API sample. I am following sentdex videos for getting started. The sample code runs perfectly, it also shows the images which are used for testing the results, but no boundaries around detected objects are shown. Just the plane image is displayed without any errors.

I'm using this code: This Github link.

This is my result after running the sample code.

another image without any detection.

What I'm missing here? The code is included in above link and there is no error logs.

Results of box, score, classes, num in that order.

  [[[ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.20880508  1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.20934391  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.20880508  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.74907303  0.14624023  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]]]
[[ 0.03587547  0.02224986  0.0186467   0.01096812  0.01003207  0.00654409
   0.00633549  0.00534311  0.0049596   0.00410213  0.00362371  0.00339186
   0.00308251  0.00303347  0.00293389  0.00277099  0.00269575  0.00266825
   0.00263925  0.00263331  0.00258657  0.00240822  0.0022581   0.00186967
   0.00184311  0.00180467  0.00177475  0.00173655  0.00172811  0.00171935
   0.00171891  0.00170288  0.00163755  0.00162967  0.00160273  0.00156545
   0.00153615  0.00140941  0.00132407  0.00131524  0.0013105   0.00129431
   0.0012582   0.0012553   0.00122365  0.00119186  0.00115651  0.00115186
   0.00112369  0.00107097  0.00105805  0.00104338  0.00102719  0.00102337
   0.00100349  0.00097762  0.00096851  0.00092741  0.00088506  0.00087696
   0.0008734   0.00084826  0.00084135  0.00083513  0.00083398  0.00082068
   0.00080583  0.00078979  0.00078059  0.00077476  0.00075448  0.00074426
   0.00074421  0.00070195  0.00068741  0.00068138  0.00067262  0.00067125
   0.00067033  0.00066035  0.00064729  0.00064205  0.00061964  0.00061794
   0.00060835  0.00060465  0.00059548  0.00059479  0.00059461  0.00059436
   0.00059426  0.00059411  0.00059406  0.00059392  0.00059365  0.00059351
   0.00059191  0.00058798  0.00058682  0.00058148]]
[[  1.   1.  18.  32.  62.  60.  63.  67.  61.  49.  31.  84.  50.  54.
   15.  44.  44.  49.  31.  56.  88.  28.  88.  52.  17.  32.  38.  75.
    3.  33.  48.  59.  35.  57.  47.  51.  19.  27.  72.   4.  84.   6.
   55.  20.  58.  65.  61.  82.  42.  34.  40.  21.  43.  64.  39.  62.
   36.  22.  79.  46.  16.  40.  41.  77.  16.  48.  78.  77.  89.  86.
   27.   8.  87.   5.  25.  70.  80.  76.  75.  67.  65.  37.   2.   9.
   73.  63.  29.  30.  69.  66.  68.  26.  71.  12.  45.  83.  13.  85.
   74.  23.]]
[ 100.]
[[[ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.00784111  0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.          1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]
  [ 0.          0.68494415  1.          1.        ]]]
[[ 0.01044297  0.0098214   0.00942165  0.00846471  0.00613666  0.00398615
   0.00357754  0.0030054   0.00255861  0.00236574  0.00232631  0.00220291
   0.00185227  0.0016354   0.0015979   0.00145072  0.00143661  0.00141369
   0.00122685  0.00118978  0.00108457  0.00104251  0.00099215  0.00096401
   0.0008708   0.00084773  0.00080484  0.00078507  0.00078378  0.00076876
   0.00072774  0.00071732  0.00071348  0.00070812  0.00069253  0.0006762
   0.00067269  0.00059905  0.00059367  0.000588    0.00056114  0.0005504
   0.00051472  0.00051057  0.00050973  0.00048486  0.00047297  0.00046204
   0.00044787  0.00043259  0.00042987  0.00042673  0.00041978  0.00040494
   0.00040087  0.00039576  0.00039059  0.00037274  0.00036831  0.00036417
   0.00036119  0.00034645  0.00034479  0.00034078  0.00033771  0.00033605
   0.0003333   0.0003304   0.0003294   0.00032326  0.00031787  0.00031773
   0.00031748  0.00031741  0.00031732  0.00031729  0.00031724  0.00031722
   0.00031717  0.00031708  0.00031702  0.00031579  0.00030416  0.00030222
   0.00029739  0.00029726  0.00028289  0.0002653   0.00026325  0.00024584
   0.00024221  0.00024156  0.00023911  0.00023335  0.00021619  0.0002001
   0.00019127  0.00018342  0.00017273  0.00015509]]
[[ 38.   1.   1.  16.  25.  38.  64.  24.  49.  56.  20.   3.  28.   2.
   48.  19.  21.  62.  50.   6.   8.   7.  67.  18.  35.  53.  39.  55.
   15.  57.  72.  52.  10.   5.  42.  43.  76.  22.  82.   4.  61.  23.
   17.  16.  87.  62.  51.  60.  36.  58.  59.  33.  31.  54.  70.  11.
   40.  79.  31.   9.  41.  77.  80.  34.  90.  89.  73.  13.  84.  32.
   63.  29.  30.  69.  66.  68.  26.  71.  12.  45.  83.  14.  44.  78.
   85.  46.  47.  19.  65.  74.  37.  27.  63.  88.  28.  81.  86.  75.
   27.  18.]]
[ 100.]

EDIT: As per suggested answers, it is working when we use faster_rcnn_resnet101_coco_2017_11_08 model. But it is more accurate and that's why slower. I want this application with high speed because I'm going to use it in real time (on webcam) object detection. So I need to use faster model (ssd_mobilenet_v1_coco_2017_11_08)

Yoho

can you show us the values for (boxes, scores, classes, num); I want to understand if any objects are detected. – Urbannal

How can I do that? @Zephro – Yoho

okay by printing the coordinates of box? – Yoho

Yes ;) print(boxes), print(scores) ,... – Urbannal

see the updates. @Zephro – Yoho

any ideas? @Zephro – Yoho

I'd like to add a me-too here. I tried exactly the same (under Linux) and have the same behavior. – Uterine

@Uterine I've raised an issue on github. Let's hope for the solution: github.com/tensorflow/models/issues/2773 – Yoho

this is also related recent question: #47239092 – Yoho

I just did a test where I picked a different model when you pick a different pretrained model: MODEL_NAME= 'faster_rcnn_resnet101_coco_2017_11_08' it does give some result. As far as I understand @Jack.Liu is correct and the scores given by the default model are way too low to pas the 0.5 threshold. – Uterine

Let me try that model then. – Yoho

yes you are right. I also got some accurate results by using that model. But know the reason. – Yoho

Answer 1

The problem is from the model: 'ssd_mobilenet_v1_coco_2017_11_08'

Solution: change to an differrent version 'ssd_mobilenet_v1_coco_11_06_2017' (this model type is the fastest one, change to other model types will make it slower and not the thing that you want)

Just change 1 line of code:

# What model to download.
MODEL_NAME = 'ssd_mobilenet_v1_coco_11_06_2017'

When I use your code, nothing is shown but when I replace it with my previous experiment model 'ssd_mobilenet_v1_coco_11_06_2017' it works fine

Biopsy

same issue.. no objects detected.. i used this model for transfer learning: ssdlite_mobilenet_v2_coco_2018_05_09 os: windows 7 python: Python 3.5.2 |Anaconda custom (64-bit)| (default, Jul 5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)] on win32 – Deplane

Sorry, I don't catch what you mean. Do you mean when you apply above code the program still doesn't detect any objects? – Biopsy

yes.. I used ssdlite_mobilenet_v2_coco_2018_05_09 and could manage to get the loss to 0.9 still no objects were detected.. now I have switched to ssd_mobilenet_v1_coca_2017_11_17 and it seems to be working.. – Deplane

i can confirm that changing the model from "2017_11_08", where nothing happens to "11_06_2017" works. I see the images with bounding boxes...Thanks, @TinLuu! – Pebble

Answer 2

As a workaround change #MODEL_NAME = 'ssd_mobilenet_v1_coco_2017_11_08' to MODEL_NAME = 'faster_rcnn_resnet101_coco_2017_11_08'.

Degradation

Answer 3

You can use older 'ssd_mobilenet_v1 ... ' and run your program completely with boxes (I run it just now and it is correct). This is a link to this older version. Hope they correct newer version soon!

Schoenberg

Answer 4

I used to have the same problem.

But a new model has been upload it recently 'ssd_mobilenet_v1_coco_2017_11_17'

I tried it and works like charm :)

Marijane

Answer 5

the function visualize_boxes_and_labels_on_image_array has the following code:

  for i in range(min(max_boxes_to_draw, boxes.shape[0])):
    if scores is None or scores[i] > min_score_thresh:

so, the score must be bigger than min_score_thresh (default 0.5), you can check whether there are some scores bigger than it.

Plural

So why no score is bigger than 0.5 even if it is detecting correctly? – Yoho

So if model 'ssd_mobilenet_v1_coco_2017_11_08' has the issue, then does it mean that training with it would also be problematic? I tried to train with it but it's stuck in first step: global_step/sec: 0. Its stuck for almost 9 hours. I am training on CPU. – Chapatti

@Yoho You can use model "faster_rcnn_resnet101_coco_2017_11_08" instead of "ssd_mobilenet_v1_coco_2017_11_08" – Plural

@Jack.Liu: It's not related to the threshold here, it's the model problem not to detect reliably – Biopsy