it requires specifying NormalizationOptions metadata to preprocess input images

Question

Yolov3-tiny-416.tflite is a tflite model for yolov3 tiny model created from yolov3-tiny.weights I had tried to use this from ML kit Vision module provided by google in android. In repo: https://github.com/googlesamples/mlkit/tree/master/android/vision-quickstart

This is the way I have load and and choose detection options for yolo v3 tiny tflite model.

LocalModel localModel = new LocalModel.Builder()
              .setAssetFilePath("yolov3-tiny-416.tflite")
              .build();
CustomObjectDetectorOptions customObjectDetectorOptions = PreferenceUtils.getCustomObjectDetectorOptionsForLivePreview(this,localModel);
cameraSource.setMachineLearningFrameProcessor(new ObjectDetectorProcessor(this,customObjectDetectorOptions));

Now, I have encounterd a error that says:

E/MobileVisionBase: Error preloading model resource
b.a.d.a.a: Failed to initialize detector. Input tensor has type kTfLiteFloat32: it requires specifying NormalizationOptions metadata to preprocess input images. 

As I know from error, I need to specify NormalizationOptions Metadata to process Image. So, How the problem can be solved? any Suggestion?

Beckman

Answer 1

Here is the custom model requirements for ML Kit custom object detection&tracking. https://developers.google.com/ml-kit/custom-models If you check the Metadata section at the bottom of the page, it has some instructions about adding NormalizationOptions Metadata.

However, the very basic requirement for ML Kit custom object detection&tracking is that the model needs to be an image classification model, while yolov3 is not.

If you want to classify more object with ML Kit, you can try one of the custom image classifier models with ML Kit tag on TFHub. https://tfhub.dev/ml-kit/collections/image-classification/1 or train your own classifier using AutoML or TFLite ModelMaker (see https://developers.google.com/ml-kit/custom-models#automl_vision_edge).

Best,

Tombaugh

I am getting my model.tflite file from Microsoft Azure Custom Vision, and am using the following in Xamarin.Android which gives me the error mentioned by the OP: ObjectDetector.ObjectDetectorOptions objoptions = ObjectDetector.ObjectDetectorOptions.InvokeBuilder().SetScoreThreshold(0.6f).Build(); ObjectDetector objdetector = ObjectDetector.CreateFromFileAndOptions(Application.Context, "model.tflite", objoptions); How do I make ObjectDetectorOptions to specify NormalizationOptions? – Defiance

By following the instructions in the link above, you should be able to get here: developers.google.com/ml-kit/custom-models#metadata , which provides instructions on "Adding metadata to TensorFlow Lite model" – Tombaugh

That link is of no use. It attempts to edit a file by the name metadata_schema.fbs, which does not exist in my project (I have never even heard of *.fbs files). As I said, I am not creating the *.tflite file, I am simply using it. If normalization means editing the *.tflite file, then how exactly do I do that (although that is not something I was expecting to need to do)? I assumed the most I would need to do was passing the appropriate parameters, otherwise why am I getting the *.tflite file from an external source? – Defiance

I haven't done it by myself, but you should be able to attach metadata to an existing tflite model. It will be easier if you can contact the owner of the model to do that if you are not familiar with that part. – Tombaugh

Stepping back tho... the model you are talking about is yolo. If I understand correctly, it is not a classification model, but an one-shot object-detection model, right? See the dev doc here: developers.google.com/ml-kit/custom-models ML Kit only supports custom classification model in the Object Detection pipeline. – Tombaugh

TFLite Task Library provides a custom Object detection solution. I never tried it by myself, but you may take a look to see if that one is more related: tensorflow.org/lite/inference_with_metadata/task_library/… – Tombaugh

I looked at tensorflow.org/lite/models/convert/… But I could not find ObjectDetectorWriter anywhere in the NuGet packages for Xamarin.Android. Where do I get ObjectDetectorWriter? – Defiance