Convert Keras model to C++ [closed]

Question

I am using Keras (with Theano) to train my CNN model. Does anyone has idea how can I use it in my C++ application? Does anyone tried something similar? I have idea to write some python code that will generate a c++ code with network functions - any suggestion on it?

I found a similar question here how to use Tensorflow Keras model in C++ but without answer.

Cowgirl

@Kala What details would you like? – Levite

@Floret Sorry, I chosen wrong reason for bounty. I would like to find more straightforward or more suitable for production way to use Keras models in C++ code. – Kala

@Kala Anything wrong with the HDF5 + system call solution? – Levite

@Floret It's hard to distribute such solution. We need to deploy all python environment with Keras and so on. – Kala

@Kala See my edited answer. – Levite

@Floret Thanks! It's good option. But I hope to find a way without dependencies on python. – Kala

@Kala I agree - like I mentioned, I would use Caffe in your situation. – Levite

Answer 1

To answer my own question and have a solution - I wrote a plain c++ solution called keras2cpp (its code available on github).

In this solution you store network architecture (in json) and weights (in hdf5). Then you can dump a network to a plain text file with provided script. You can use obtained text file with network in pure c++ code. There are no dependencies on python libraries or hdf5. It should work for theano and tensorflow backend.

Cowgirl

Sorry, didn't realize you wanted to deploy an already trained model. In that case, this is a good solution. – Levite

Answer 2

I found myself in a similar situation but needed to not only support forward passes of sequential Keras models in C++ but also of more complex models build with the functional API.

So I wrote a new library called frugally-deep. You can find it on GitHub and it is published under the MIT License: https://github.com/Dobiasd/frugally-deep

Additionally to supporting many common layer types it can keep up with (and sometimes even beat) the performance of TensorFlow on a single CPU. You can find up-to-date benchmark results for some common model in the repo.

By automatic testing frugally-deep guarantees that the output of a model used with it in C++ is exactly the same as if run with Keras in Python.

Conspecific

Does your library support an inference of a model trained using tensorflow.keras? – Freewheel

@off99555 Yes it does. :) – Conspecific

Answer 3

If your keras model is trained using tensorflow backend, you can save the keras model as a tensorflow model following this code: https://github.com/amir-abdi/keras_to_tensorflow

Here is a shorter version of the code:

from keras import backend as K
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io

weight_file_path = 'path to your keras model'
net_model = load_model(weight_file_path)
sess = K.get_session()

constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), 'name of the output tensor')
graph_io.write_graph(constant_graph, 'output_folder_path', 'output.pb', as_text=False)
print('saved the constant graph (ready for inference) at: ', osp.join('output_folder_path', 'output.pb'))

Catching

Answer 4

You Can try this one https://github.com/gosha20777/keras2cpp

Keras2cpp is a small library for running trained Keras models from a C++ application without any dependencies.

Supported Keras layers: - Dense - Convolution1D - Convolution2D - Convolution3D - Flatten - ELU - Activation - MaxPooling2D - Embedding - LocallyConnected1D - LocallyConnected2D - LSTM - GRU - CNN - BatchNormalization

Supported activation: - linear - relu - softplus - tanh - sigmoid - hard_sigmoid - elu - softsign - softmax

Design goals:

• Compatibility with networks generated by Keras using TensorFlow backend.
• CPU only.
• No external dependencies, standard library, C++17.
• Model stored in memory.

Doley

Welcome to Stack Overflow! While we appreciate your intent of contributing with an answer, please make sure that the question is on-topic for the site first. This is a request for an off-site resource, and therefore should not be answered See: Should one advise on off topic questions? – Danzig

Hi, welcome to Stack Overflow. It looks like you're linking to your own GitHub library. You need to properly disclose that fact as your answer may be liable to deletion if you don't. Please take a moment to read SO's guidance on self-promotion – Underwrite

Answer 5

I had a similar need--I wanted to embed Keras models in a C++ application--and decided to write my own library: Kerasify

Design goals of Kerasify:

• Compatibility with image processing Sequential networks generated by Keras using Theano backend. (Could work with Tensorflow if you switch around matrix col/row ordering).
• No external dependencies, standard library, C++11 features OK.
• Model stored on disk in binary format that can be quickly read.
• Model stored in memory in contiguous block for better cache performance.
• Doesn't throw exceptions, returns only bool on error.
• CPU only, no GPU

Example code, unit tests, etc. at the github link. It's not fully complete, it only supports the narrow subset of Keras functions I'm using, but it should be extensible with a little effort.

Jeer

Answer 6

The solutions found here are quite good, but if your model has some different types of layers not supported by these libraries, I would recommend doing the following:

• Converting the Keras model to a tensorflow model.
• Freeze the model and use Tranform graph tool provided by tensorflow (you'll have to build it from source with bazel)
• Compile the C++ API tensorflow library to use it in your project.
• Use the C++ API tensorflow library and link the libraries to your project.

If you want to use a something differentcompiler than bazel (like g++ for example) you can follow this great tuturial:

http://tuatini.me/building-tensorflow-as-a-standalone-project/

Keratin

Small addition: Bazel is not a compiler, it is a build tool (which may use a compiler like g++ but not replace it) – Slim

Good catch, edited the answer. – Keratin

Answer 7

The easiest way is probably to make a system call to a Python script that writes the predictions to a binary or HDF5 file, which can be read in from C++. You can also directly integrate Python into C++.

If you need to deploy and distribute this easily, you can look into self-contained Python installations like Anaconda, but your best bet may be to avoid Keras and use the C++ interface to Caffe or Tensorflow. I wouldn't recommend Tensorflow since using it from C++ isn't standard; see this discussion. Caffe is arguably the second most-popular deep learning library so you can't really go wrong.

Floret