Export a basic Tensorflow model to Google Cloud ML

Question

I am trying to export my local tensorflow model to use it on Google Cloud ML and run predictions on it.

I am following the tensorflow serving example with mnist data. There is quite a bit of difference in the way they have processed and used their input/output vectors and it is not what you find in typical examples online.

I am unsure how to set the parameters of my signatures :

model_exporter.init(
    sess.graph.as_graph_def(),
    init_op = init_op,
    default_graph_signature = exporter.classification_signature(
        input_tensor = "**UNSURE**" ,
        scores_tensor = "**UNSURE**"),
    named_graph_signatures = {
        'inputs' : "**UNSURE**",
        'outputs': "**UNSURE**"
    }

    )
model_exporter.export(export_path, "**UNSURE**", sess)

Here is the rest of my code:

import sys
import tensorflow as tf
from tensorflow.contrib.session_bundle import exporter

import numpy as np
from newpreprocess import create_feature_sets_and_labels

train_x,train_y,test_x,test_y = create_feature_sets_and_labels()

x = tf.placeholder('float', [None, 13])
y = tf.placeholder('float', [None, 1])

n_nodes_hl1 = 20
n_nodes_hl2 = 20

n_classes = 1
batch_size = 100

def neural_network_model(data):

    hidden_1_layer = {'weights':tf.Variable(tf.random_normal([13, n_nodes_hl1])),
                      'biases':tf.Variable(tf.random_normal([n_nodes_hl1]))}

    hidden_2_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl1, n_nodes_hl2])),
                      'biases':tf.Variable(tf.random_normal([n_nodes_hl2]))}

    output_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl2, n_classes])),
                    'biases':tf.Variable(tf.random_normal([n_classes]))}


    l1 = tf.add(tf.matmul(data, hidden_1_layer['weights']), hidden_1_layer['biases'])
    l1 = tf.tanh(l1)

    l2 = tf.add(tf.matmul(l1, hidden_2_layer['weights']), hidden_2_layer['biases'])
    l2 = tf.tanh(l2)

    output = tf.add(tf.matmul(l2, output_layer['weights']), output_layer['biases'])
    return output



def train_neural_network(x):
    output = neural_network_model(x)
    cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(output, y))
    optimizer = tf.train.AdamOptimizer(0.003).minimize(cost)

    hm_epochs = 700

    with tf.Session() as sess:
        sess.run(tf.initialize_all_variables())

        for epoch in range(hm_epochs):
            epoch_loss = 0
            i = 0
            while i < len(train_x):
                start = i
                end = i + batch_size
                batch_x = np.array(train_x[start:end])
        batch_y = np.array(train_y[start:end])

        _, c = sess.run([optimizer, cost], feed_dict={x: batch_x,
                                              y: batch_y})
        epoch_loss += c
        i+=batch_size

            print('Epoch', epoch, 'completed out of', hm_epochs, 'loss:', epoch_loss/(len(train_x)/batch_size))


        prediction = tf.sigmoid(output)
        predicted_class = tf.greater(prediction,0.5)
        correct = tf.equal(predicted_class, tf.equal(y,1.0))
        accuracy = tf.reduce_mean( tf.cast(correct, 'float') )

        print('Accuracy:', accuracy.eval({x: test_x, y: test_y}))

        export_path = "~/Documents/cloudcomputing/Project/RNN_timeseries/"
        print ("Exporting trained model to %s", %export_path)
        init_op = tf.group(tf.initialize_all_tables(), name="init_op")
        saver = tf.train.Saver(sharded = True)
        model_exporter = exporter.Exporter(saver)
        model_exporter.init(
            sess.graph.as_graph_def(),
            init_op = init_op,
            default_graph_signature = exporter.classification_signature(
                input_tensor = ,
                scores_tensor = ),
            named_graph_signatures = {
                'inputs' : ,
                'outputs': 
            }

            )
        model_exporter.export(export_path, tf.constant(1), sess)
        print("Done exporting!")



train_neural_network(x)

What exactly are the steps to upload and use this on Google Cloud ML? Their walkthroughs seem to be for models trained on the cloud itself and not on local machines.

Fiscal

While the walkthrough demonstrates training on the cloud, you can follow most of the same steps to train locally then deploy to the cloud. In either case, you'll end up with a directory containing an exported model and you just need to point to that directory when deploying the model (if you're not using gcloud, you'll need to make sure to copy the model to GCS). – Progeny

Answer 1

Tensorflow Serving and Google Cloud ML are two different things, don't mix them up. Cloud ML is a fully managed solution (ML as a service), whereas TF Serving requires you to set up and maintain your infrastructure - it's just a server. They are unrelated and have different requirements in input/output handling.

The guide that you should follow is this one. Instead of using graph signatures you add inputs and outputs into collections. The changes in your code would then be something like this:

import sys
import tensorflow as tf
from tensorflow.contrib.session_bundle import exporter

import numpy as np
from newpreprocess import create_feature_sets_and_labels
import json 
import os 

train_x,train_y,test_x,test_y = create_feature_sets_and_labels()

n_nodes_hl1 = 20
n_nodes_hl2 = 20
n_classes = 1
batch_size = 100

x = tf.placeholder('float', [None, 13])
y = tf.placeholder('float', [None, 1])
keys_placeholder = tf.placeholder(tf.int64, shape=(None,))

keys = tf.identity(keys_placeholder)

def neural_network_model(data):
    hidden_1_layer = {'weights':tf.Variable(tf.random_normal([13, n_nodes_hl1])),
                      'biases':tf.Variable(tf.random_normal([n_nodes_hl1]))}
    hidden_2_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl1, n_nodes_hl2])),
                      'biases':tf.Variable(tf.random_normal([n_nodes_hl2]))}
    output_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl2, n_classes])),
                    'biases':tf.Variable(tf.random_normal([n_classes]))}
    l1 = tf.add(tf.matmul(data, hidden_1_layer['weights']), hidden_1_layer['biases'])
    l1 = tf.tanh(l1)
    l2 = tf.add(tf.matmul(l1, hidden_2_layer['weights']), hidden_2_layer['biases'])
    l2 = tf.tanh(l2)
    output = tf.add(tf.matmul(l2, output_layer['weights']), output_layer['biases'])
    return output

output = neural_network_model(x)
prediction = tf.sigmoid(output)
predicted_class = tf.greater(prediction,0.5)


inputs = {'key': keys_placeholder.name, 'x': x.name}
tf.add_to_collection('inputs', json.dumps(inputs))

outputs = {'key': keys.name,
           'prediction': predicted_class.name}
tf.add_to_collection('outputs', json.dumps(outputs))


def train_neural_network(x):
    cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(output, y))
    optimizer = tf.train.AdamOptimizer(0.003).minimize(cost)
    hm_epochs = 700

    with tf.Session() as sess:
        sess.run(tf.initialize_all_variables())
        for epoch in range(hm_epochs):
            epoch_loss = 0
            i = 0
            while i < len(train_x):
                start = i
                end = i + batch_size
                batch_x = np.array(train_x[start:end])
                batch_y = np.array(train_y[start:end])

                _, c = sess.run([optimizer, cost], feed_dict={x: batch_x,
                                              y: batch_y})
                epoch_loss += c
                i+=batch_size
            print('Epoch', epoch, 'completed out of', hm_epochs, 'loss:', epoch_loss/(len(train_x)/batch_size))

        correct = tf.equal(predicted_class, tf.equal(y,1.0))
        accuracy = tf.reduce_mean( tf.cast(correct, 'float') )
        print('Accuracy:', accuracy.eval({x: test_x, y: test_y}))

        export_path = "~/Documents/cloudcomputing/Project/RNN_timeseries/"
        print ("Exporting trained model to %s", %export_path)
        init_op = tf.group(tf.initialize_all_tables(), name="init_op")

        saver = tf.train.Saver(sharded = True)
        saver.save(sess, os.path.join(export_path, 'export'))

        print("Done exporting!")

train_neural_network(x)

I moved some things in your code a little (and haven't actually tested it), but that should give you a starting point.

Descendible

After running your code, I obtained checkpoint, export.meta and export-00000-of-00001. Is the last one the graph file or the 1st one? – Fiscal

export.meta contains the definition of graph operations and constants, the other the trained values of variables. If you didn't set sharder=True it would be without the numbers, but that makes little difference. Checkpoint is kind of like a pointer. Anyway, you can just upload all of them to a bucket on Storage and it will work ;) – Descendible

Oh. Actually I was trying that but I am facing an error with the deploy part. When I try to create a version as mentioned here: cloud.google.com/ml/docs/how-tos/deploying-models - I run into an error: Error Sorry, there's a problem. If you have entered information, check it and try again. Otherwise, the problem might clear up on its own, so check back later. – Fiscal

@Progeny I got it to work sometime back. I used the command line option and uploaded my model files to a folder in my bucket. Trying to run the predictions now but not sure how that works. – Fiscal

I uploaded my json line : {"key":0, "x" : [159.220001,159.929993,158.850006,159.800003,2256400,159.800003,153.94036155,162.320475227,145.560247873,159.448001,157.2819584,153.4395999,218.990005]} to a file and uploaded that to a bucket. Is this the correct way to upload input data for prediction? – Fiscal

For debugging purposes, gcloud ml beta predict should work. The file you have looks correct. It does not need to be on GCS. You can also use gcloud ml beta local predict to debug things locally before deploying to the cloud. If you plan to use things in production, you'll probably want to use the HTTP API, but in that case, the request body is slightly different: {"instances": {"key":0, "x": [159....005]}} – Progeny

I tried this command: gcloud beta ml predict --model=new_model --instances=gs://bucket_name/data/predict_sample.tensor.json. But I got the following error: usage: gcloud beta ml predict --model=MODEL [optional flags] ERROR: (gcloud.beta.ml.predict) one of the arguments --json-instances --text-instances is required – Fiscal

Okay. I realized that I wasn't using --json-instances or --text-instances while calling my json file. But when I use --json-instances and point to my json file in my GCS - --json-instances=gs://bucket-name-ml/data/predict_sample.tensor.json I get the error -- ERROR: gcloud crashed (IOError): [Errno 2] No such file or directory: 'gs://bucket-name-ml/data/predict_sample.tensor.json' – Fiscal

The json file should be on the machine (your laptop or VM, whatever you're using) you're running the command from, not in a bucket. – Descendible

Finally got it to run. All my predictions are false however. And my labels are supposed to be 1 or 0. – Fiscal

@RobertLacok Any idea why all responses are false? – Fiscal

@VineetKaushik does this help? #40929723 – Progeny