deploying the Tensorflow model in Python
Asked Answered
S

1

5

Need help in implementing the Tensorflow model in real time. While I am training everything is working fine but when I move on for a realtime forecast or prediction, the output what I received flunked. I do not know why is this happening. I used the reference of teh code from here: https://www.kaggle.com/raoulma/ny-stock-price-prediction-rnn-lstm-gru/notebook And tried to implement or deploy using the same code with few changes.

See the following code:

import numpy as np
import pandas as pd
import sklearn
import sklearn.preprocessing
import datetime
import os
import tensorflow as tf

df = pd.read_csv("Realtime_Values.csv", index_col = 0)
df.info()
def load_data(stock,seq_len):

    data_raw = stock.as_matrix() # convert to numpy array
    data = []

    for index in range(len(data_raw) - seq_len): 
        data.append(data_raw[index: index + seq_len])
    #print(len(data))
    data = np.array(data);

    x_forecast = data[:,:-1,:]
    return x_forecast

def normalize_data(df):
    cols = list(df.columns.values)
    min_max_scaler = sklearn.preprocessing.MinMaxScaler()
    df = pd.DataFrame(min_max_scaler.fit_transform(df.values))
    df.columns = cols
    return df
model_path ="modelsOHLC"
seq_len = 9
# parameters
n_steps = seq_len-1 
n_inputs = 4
n_neurons = 100 
n_outputs = 4
n_layers = 4
learning_rate = 0.01
batch_size = 10
n_epochs = 1000
tf.reset_default_graph()

X = tf.placeholder(tf.float32, [None, n_steps, n_inputs])
y = tf.placeholder(tf.float32, [None, n_outputs])
layers = [tf.contrib.rnn.BasicRNNCell(num_units=n_neurons, activation=tf.nn.elu)
          for layer in range(n_layers)]
multi_layer_cell = tf.contrib.rnn.MultiRNNCell(layers)
rnn_outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)

stacked_rnn_outputs = tf.reshape(rnn_outputs, [-1, n_neurons]) 
stacked_outputs = tf.layers.dense(stacked_rnn_outputs, n_outputs)
outputs = tf.reshape(stacked_outputs, [-1, n_steps, n_outputs])
outputs = outputs[:,n_steps-1,:] # keep only last output of sequence

loss = tf.reduce_mean(tf.square(outputs - y)) # loss function = mean squared error 
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate) 
training_op = optimizer.minimize(loss)
saver = tf.train.Saver()
sess  =tf.Session()
sess.run(tf.global_variables_initializer())    
if(tf.train.checkpoint_exists(tf.train.latest_checkpoint(model_path))):
        saver.restore(sess, tf.train.latest_checkpoint(model_path))
df = normalize_data(df)
x_forecast = load_data(df,seq_len)
y_forecast_pred = sess.run(outputs, feed_dict={X: x_forecast})
print(y_forecast_pred)

Can anyone help me in getting the above code run in real time without any issues?

Sianna answered 29/10, 2018 at 10:42 Comment(7)
I don't see any code for training loop. Can you add your training loop code?Swallowtail
Dear, please read my question. I have a kaggle link available for it. You can see my code from there. The same code I have used for training, validation, and testing. But the only problem is with the implementation in real time scenario. Hence, have mentioned what I tried.Sianna
What is the exact problem you're having? Is it just that your results aren't accurate enough or are you getting an unexpected output in some other way? It's not really possible to answer your question without knowing what you want.Administer
Yes. In real time scenario, I am not getting the output as I used to get in the training and even in testing. Please let me know what is wrong with my implementation.Sianna
check out this article: towardsdatascience.com/deploy-tensorflow-models-9813b5a705d5Sackbut
I tried it and even had a conversation with the writer on github. It didn't worked for me. And the writer is not even supportive to help. Guess he is damn busy to have a look at someones issue.Sianna
Are your training results similar to the ones in the article? and what do your actual output results look like?Ar
U
6

There is a possibility that the code failed to find the saved weights when program trains the model; thus the predictions are being generated at an untrained state. Your code for training model is:

if (tf.train.checkpoint_exists(tf.train.latest_checkpoint(model_path))):
    saver.restore(sess, tf.train.latest_checkpoint(model_path))

To fix this problem:

  • Add a debugging code such as print("checkpoint exists!")

  • Place breakpoint through a debugger before or after save.restore(...) to find a checkpoint to restore from.

  • Look at the model_path to ensure your checkpoints are saved correctly.

Udell answered 6/11, 2018 at 14:51 Comment(1)
I don't think that the code is not finding the saved model. Because if the code didn't have loaded the saved model then how is it predicting the future frames?Sianna

© 2022 - 2024 — McMap. All rights reserved.