Load model with ML.NET saved with keras

Question

I have a Neural Network implemented in Python with Keras. Once I have trained it I have exported the model and I have got two files: model.js and model.h5. Now I want to classify in real time inside a .NET project and I want to use the trained Neural Network for it.

Is there a way in ML.NET of loading the model and trained weights exported with python into a model object?

I have seen in the documentation[1] that a previous saved model can be loaded, but apparently is storage in a .zip and I could not find the format (maybe to make a script that takes the model from python and 'translate' it to the ML.NET model.

Apparently the hdf5 format is a standard[2], there is a way to load it with ML.NET?

[1] https://learn.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/consuming-model-ml-net

[2] https://support.hdfgroup.org/HDF5/doc1.6/UG/10_Datasets.html

Gag

I think you'd have to convert it to an ONNX model and read it that way. – Knackwurst

Answer 1

ML.net supports ONNX models, as in this example.

You can convert your keras model to ONNX model via WinMLTools

Accord

Answer 2

I have been through the same path and I would strongly suggest to use Keras2onnx library package for Python to convert your Keras modules to "onnx" format first. The simple code I am using is as follows:

reconstructed_model = keras.models.load_model("<your location>\\my_model")

import onnx
import keras2onnx

model_name_onnx = "model.onnx"

onnx_model = keras2onnx.convert_keras(reconstructed_model, reconstructed_model.name)

onnx.save_model(onnx_model, model_name_onnx)

On the C# side, in terms of how to process the data (i.e. images) and make a prediction, follow the Microsoft example (https://learn.microsoft.com/en-us/dotnet/machine-learning/tutorials/object-detection-onnx)

You can use ML.NET with dotNet Core or Net framework > 4.7. Here is an excerpt as how to read image files and make predictions running the model.onnx.

///First, load the data into an IDataView.
   IEnumerable<ImageNetData> images = ImageNetData.ReadFromFile(imagesFolder);
   IDataView imageDataView = mlContext.Data.LoadFromEnumerable(images);

   var modelScorer = new OnnxModelScorer(imagesFolder, modelFilePath, mlContext);

// Use model to score data
   IEnumerable<float[]> probabilities = modelScorer.Score(imageDataView);

ImageNetData class basically reads images at imageFolder by filtering image files (from a directory) and identifies them with two properties, a label which is the filename and the ImagePath which is the path of the imagefile. OnnxModelScorer class runs the Load Model and Prediction methods.

That is all you need for your python Keras generated model predictions from C#.

Clay