TensorFlow: "Attempting to use uninitialized value" in variable initialization

Question

I am trying to implement multivariate linear regression in Python using TensorFlow, but have run into some logical and implementation issues. My code throws the following error:

Attempting to use uninitialized value Variable
Caused by op u'Variable/read'

Ideally the weights output should be [2, 3]

def hypothesis_function(input_2d_matrix_trainingexamples,
                        output_matrix_of_trainingexamples,
                        initial_parameters_of_hypothesis_function,
                        learning_rate, num_steps):
    # calculate num attributes and num examples
    number_of_attributes = len(input_2d_matrix_trainingexamples[0])
    number_of_trainingexamples = len(input_2d_matrix_trainingexamples)

    #Graph inputs
    x = []
    for i in range(0, number_of_attributes, 1):
        x.append(tf.placeholder("float"))
    y_input = tf.placeholder("float")

    # Create Model and Set Model weights
    parameters = []
    for i in range(0, number_of_attributes, 1):
        parameters.append(
            tf.Variable(initial_parameters_of_hypothesis_function[i]))

    #Contruct linear model
    y = tf.Variable(parameters[0], "float")
    for i in range(1, number_of_attributes, 1):
        y = tf.add(y, tf.multiply(x[i], parameters[i]))

    # Minimize the mean squared errors
    loss = tf.reduce_mean(tf.square(y - y_input))
    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
    train = optimizer.minimize(loss)

    #Initialize the variables
    init = tf.initialize_all_variables()

    # launch the graph
    session = tf.Session()
    session.run(init)
    for step in range(1, num_steps + 1, 1):
        for i in range(0, number_of_trainingexamples, 1):
            feed = {}
            for j in range(0, number_of_attributes, 1):
                array = [input_2d_matrix_trainingexamples[i][j]]
                feed[j] = array
            array1 = [output_matrix_of_trainingexamples[i]]
            feed[number_of_attributes] = array1
            session.run(train, feed_dict=feed)

    for i in range(0, number_of_attributes - 1, 1):
        print (session.run(parameters[i]))

array = [[0.0, 1.0, 2.0], [0.0, 2.0, 3.0], [0.0, 4.0, 5.0]]
hypothesis_function(array, [8.0, 13.0, 23.0], [1.0, 1.0, 1.0], 0.01, 200)

Thrombus

What line do you get the exception on? – Bonnibelle

@Daniel Slater at line :- parameters.append(tf.Variable(initial_parameters_of_hypothesis_function[i])) – Thrombus

OK, is initial_parameters_of_hypothesis_function an array of tf.variable? If so that is your problem. – Bonnibelle

Yes at very last line it is [1.0,1.0,1.0] What should be then ? – Thrombus

Can you include the code to generate the initial_parameters_of_hypothesis_function in your sample? (Also to make it smaller removing everything after the line with the exception) – Bonnibelle

At very end of code snippet there is code to initialize initial_parameters_of_hypothesis_function – Thrombus

If the issue is on gcloud - check this answer: https://mcmap.net/q/354752/-unable-to-make-predictions-on-google-cloud-ml-whereas-same-model-is-working-on-the-local-machine – Grosgrain

Answer 1

It's not 100% clear from the code example, but if the list initial_parameters_of_hypothesis_function is a list of tf.Variable objects, then the line session.run(init) will fail because TensorFlow isn't (yet) smart enough to figure out the dependencies in variable initialization. To work around this, you should change the loop that creates parameters to use initial_parameters_of_hypothesis_function[i].initialized_value(), which adds the necessary dependency:

parameters = []
for i in range(0, number_of_attributes, 1):
    parameters.append(tf.Variable(
        initial_parameters_of_hypothesis_function[i].initialized_value()))

Huai

That worked but now it gives error :- TypeError: Cannot interpret feed_dict key as Tensor: Can not convert a int into a Tensor. at line session.run(train, feed_dict=feed) – Thrombus

The error message tells you what's wrong: the keys of the feed dictionary must be Tensor objects (typically tf.placeholder() tensors) and not int values. You probably want to replace feed[j] = array with feed[x[j]] = array. – Huai

Running the train op (returned by tf.train.GradientDescentOptimizer().minimize(loss)) while feeding in different training examples seems like a good start. If you have more specific questions, feel free to ask another question! – Huai

although now my code runs correct , it gives same value of parameters as initial value with which i initialize – Thrombus

This can happen if your parameters are stuck in a local minimum. A common error is to initialize all of your weights to zero - instead you should initialize them randomly (using e.g. tf.truncated_normal()). – Huai

Shouldn't be that parameters.append(tf.Variable(initial_parameters_of_hypothesis_function[i]).initialized_value()) as initial_parameters_of_hypothesis_function is a list in OP. Also the tensorflow.org/versions/r0.7/api_docs/python/… refers to an older version of the API - what would be the right way to this now ? – Kansas

Answer 2

Run this:

init = tf.global_variables_initializer()
sess.run(init)

Or (depending on the version of TF that you have):

init = tf.initialize_all_variables()
sess.run(init)

Abnormity

init = tf.global_variables_initializer() – Mages

Excusde me initialize_all_variables is deprecated - tensorflow.org/api_docs/python/tf/initialize_all_variables – Kansas

hmm, why is this not the top answer? – Ecker

@Ecker exactly! even I have the same question – Luttrell

Answer 3

There is another the error happening which related to the order when calling initializing global variables. I've had the sample of code has similar error FailedPreconditionError (see above for traceback): Attempting to use uninitialized value W

def linear(X, n_input, n_output, activation = None):
    W = tf.Variable(tf.random_normal([n_input, n_output], stddev=0.1), name='W')
    b = tf.Variable(tf.constant(0, dtype=tf.float32, shape=[n_output]), name='b')
    if activation != None:
        h = tf.nn.tanh(tf.add(tf.matmul(X, W),b), name='h')
    else:
        h = tf.add(tf.matmul(X, W),b, name='h')
    return h

from tensorflow.python.framework import ops
ops.reset_default_graph()
g = tf.get_default_graph()
print([op.name for op in g.get_operations()])
with tf.Session() as sess:
    # RUN INIT
    sess.run(tf.global_variables_initializer())
    # But W hasn't in the graph yet so not know to initialize 
    # EVAL then error
    print(linear(np.array([[1.0,2.0,3.0]]).astype(np.float32), 3, 3).eval())

You should change to following

from tensorflow.python.framework import ops
ops.reset_default_graph()
g = tf.get_default_graph()
print([op.name for op in g.get_operations()])
with tf.Session() as 
    # NOT RUNNING BUT ASSIGN
    l = linear(np.array([[1.0,2.0,3.0]]).astype(np.float32), 3, 3)
    # RUN INIT
    sess.run(tf.global_variables_initializer())
    print([op.name for op in g.get_operations()])
    # ONLY EVAL AFTER INIT
    print(l.eval(session=sess))

Blaylock

Answer 4

Normally there are two ways of initializing variables, 1) using the sess.run(tf.global_variables_initializer()) as the previous answers noted; 2) the load the graph from checkpoint.

You can do like this:

sess = tf.Session(config=config)
saver = tf.train.Saver(max_to_keep=3)
try:
    saver.restore(sess, tf.train.latest_checkpoint(FLAGS.model_dir))
    # start from the latest checkpoint, the sess will be initialized 
    # by the variables in the latest checkpoint
except ValueError:
    # train from scratch
    init = tf.global_variables_initializer()
    sess.run(init)

And the third method is to use the tf.train.Supervisor. The session will be

Create a session on 'master', recovering or initializing the model as needed, or wait for a session to be ready.

sv = tf.train.Supervisor([parameters])
sess = sv.prepare_or_wait_for_session()

Manufacture

Answer 5

I want to give my resolution, it work when i replace the line [session = tf.Session()] with [sess = tf.InteractiveSession()]. Hope this will be useful to others.

Aurie

Thanks, this was indeed helpful for me while running on Jupyter Notebook. Can explain why does it work though? – Decedent

@Decedent Interactive Session is used for the entire instance of the notebook. So your session is always on. However, if we use tensorflow.Session() it is only for a specific region. For instance we use with keyword like (with tf.Session as sess:) – Luttrell

Answer 6

run both:

sess.run(tf.global_variables_initializer())

sess.run(tf.local_variables_initializer())

Sinuosity