OpenVino converted model not returning same score values as original model (Sigmoid)

I've converted a Keras model for use with OpenVino. The original Keras model used sigmoid to return scores ranging from 0 to 1 for binary classification. After converting the model for use with OpenVino, the scores are all near 0.99 for both classes but seem slightly lower for one of the classes.

For example, test1.jpg and test2.jpg (from opposite classes) yield scores of 0.00320357 and 0.9999, respectively.

With OpenVino, the same images yield scores of 0.9998982 and 0.9962392, respectively.

Edit* One suspicion is that the input array is still accepted by the OpenVino model but is somehow changed in shape or "scrambled" and therefore is never a match for class one? In other words, if you fed it random noise, the score would also always be 0.9999. Maybe I'd have to somehow get the OpenVino model to accept the original shape (1,180,180,3) instead of (1,3,180,180) so I don't have to force the input into a different shape than the one the original model accepted? That's weird though because I specified the shape when making the xml and bin for openvino:

python3 /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo_tf.py --saved_model_dir /Users/.../Desktop/.../model13 --output_dir /Users/.../Desktop/... --input_shape=\[1,180,180,3]

However, I know from error messages that the inference engine is expecting (1,3,180,180) for some unknown reason. Could that be the problem? The other suspicion is something wrong with how the original model was frozen. I'm exploring different ways to freeze the original model (keras model converted to pb) in case the problem is related to that.

I checked to make sure the Sigmoid activation function is being used in the OpenVino implementation (same activation as the Keras model) and it looks like it is. Why, then, are the values not the same? Any help would be much appreciated.

The code for the OpenVino inference is:

import openvino
from openvino.inference_engine import IECore, IENetwork 
from skimage import io
import sys
import numpy as np
import os

def loadNetwork(model_xml, model_bin):

    ie = IECore() 

    network =  ie.read_network(model=model_xml, weights=model_bin)

    input_placeholder_key = list(network.input_info)[0]
    input_placeholder = network.input_info[input_placeholder_key]

    output_placeholder_key = list(network.outputs)[0]
    output_placeholder = network.outputs[output_placeholder_key]

    return network, input_placeholder_key, output_placeholder_key

batch_size = 1
channels = 3
IMG_HEIGHT = 180
IMG_WIDTH = 180

#loadNetwork('saved_model.xml','saved_model.bin')

image_path = 'test.jpg'

def load_source(path_to_image):
    image = io.imread(path_to_image)
    img = np.resize(image,(180,180))
    return img

img_new = load_source('test2.jpg')

#Batch?

def classify(image):
    device = 'CPU'
    network, input_placeholder_key, output_placeholder_key = loadNetwork('saved_model.xml','saved_model.bin')
    ie = IECore() 
    exec_net = ie.load_network(network=network, device_name=device)
    res = exec_net.infer(inputs={input_placeholder_key: image})
    print(res)
    res = res[output_placeholder_key]
    return res

result = classify(img_new)
print(result)
result = result[0]
top_result = np.argmax(result)
print(top_result)
print(result[top_result])

And the result:

{'StatefulPartitionedCall/model/dense/Sigmoid': array([[0.9962392]], dtype=float32)}
[[0.9962392]]
0
0.9962392

Generally, Tensorflow is the only network with the shape NHWC while most others use NCHW. Thus, the OpenVINO Inference Engine satisfies the majority of networks and uses the NCHW layout. Model must be converted to NCHW layout in order to work with Inference Engine.

The conversion of the native model format into IR involves the process where the Model Optimizer performs the necessary transformation to convert the shape to the layout required by the Inference Engine (N,C,H,W). Using the --input_shape parameter with the correct input shape of the model should suffice.

Besides, most TensorFlow models are trained with images in RGB order. In this case, inference results using the Inference Engine samples may be incorrect. By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with --reverse_input_channels argument.

I suggest you validate this by inferring your model with the Hello Classification Python Sample instead since this is one of the official samples provided to test the model's functionality.

You may refer to this "Intel Math Kernel Library for Deep Neural Network" for deeper explanation regarding the input shape.

Recommended topics

Hot tags