Tensorflow 2 Object Detection with OpenCV C++

Question

I have trained SSD ResNet V1 model using Tensorflow 2 Object Detection API. Then I wanted to use this model with OpenCV in C++ code.

First of all, after training I had three files:

• checkpoint
• ckpt-101.data-00000-of-00001
• ckpt-101.index

Note that I don't have .meta file because it wasn't generated.

Then I created SavedModel from these files using exporter_main_v2.py script that is in Object Detection API:

python3 exporter_main_v2.py input_type=image_tensor --pipeline_config_path /path/to/pipeline.config --trained_checkpoint_dir=/path/to/checkouts --output_directory=/path/to/output/directory

Having run this script I got saved_model.pb

I tried to use this file in OpenCV in such way:

cv::dnn::Net net = cv::dnn::readNetFromTensorflow("/path/to/saved_model.pb");

But I got the following error:

OpenCV(4.2.0) /home/andrew/opencv/modules/dnn/src/tensorflow/tf_io.cpp:42: error: (-2:Unspecified error) FAILED: ReadProtoFromBinaryFile(param_file, param). Failed to parse GraphDef file: /home/andrew/Documents/tensorflow_detection/workspace/pb_model/saved_model/saved_model.pb in function 'ReadTFNetParamsFromBinaryFileOrDie'

Then I tried to freeze saved_model.pb. But, as I understood, it is impossible in TF2.x because TF2.x doesn't support Sessions and Graphs. Also I don't have .pbtxt file.

My question: is it possible to use models trained with TF2 Object Detection API in OpenCV C++?

I will be grateful if you help me to solve this problems or give any useful advices.

Bui

Answer 1

It is possible to use Tensorflow 2 models with the Object Detection API and Opencv as said in the dedicated wiki : https://github.com/opencv/opencv/wiki/TensorFlow-Object-Detection-API

So far they are more models compatible with Tensorflow 1 but it should be okay for a SSD. To freeze your graph you have to do :

import tensorflow as tf

from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2

loaded = tf.saved_model.load('my_model')
infer = loaded.signatures['serving_default']

f = tf.function(infer).get_concrete_function(input_1=tf.TensorSpec(shape=[None, 224, 224, 3], dtype=tf.float32))
f2 = convert_variables_to_constants_v2(f)
graph_def = f2.graph.as_graph_def()

# Export frozen graph
with tf.io.gfile.GFile('frozen_graph.pb', 'wb') as f:
   f.write(graph_def.SerializeToString())

As said in this comment in OpenCV Github issues : https://github.com/opencv/opencv/issues/16582#issuecomment-603819498

You will then probably need to use the tf_text_graph_ssd.py provided in OpenCV wiki to generate the text graph representation of the frozen model and that'd be it!

Frig

I have used this code, but it didn't work for me. I get error ValueError: Input 1 of node StatefulPartitionedCall was passed float from 53831:0 incompatible with expected resource. Do you know how to solve it? Also as I mentioned in question I want to use OpenCV C++, not Python. – Pop

The c++ shouldn't cause any problem. As for the error raised I don't know sorry.. You could try launching the script to freeze the model with Tensorflow 1.15 or lower. – Frig

Answer 2

Tensorflow 2 no longer supports sessions so you can’t easily export your model as a frozen graph. I found this which solved the issues I had with using Tensorflow Object Detection models with opencv. Hopefully this will help.

Clamber

Thank you for response, I will read this article. Could you tell me please which exact version of Tensorflow did you use for training? – Pop

Trained on colab VERSION: 1.15.2 – Clamber