Duplicate node name in graph: 'conv2d_0/kernel/Adam'

Question

I just saved a model, by that code:

def train():    
with tf.Session() as sess:
    saver = tf.train.Saver(max_to_keep = 2)
    Loss = myYoloLoss([Scale1,Scale2,Scale3],[Y1, Y2 ,Y3])
    opt = tf.train.AdamOptimizer(2e-4).minimize(Loss)
    init = tf.global_variables_initializer()
    sess.run(init)
    imageNum = 0
    Num = 0
    while(1):
        #get batchInput
        batchImg,batchScale1,batchScale2,batchScale3 = getBatchImage(batchSize = BATCHSIZE)
        for epoch in range(75):
            _ , epochloss = sess.run([opt,Loss],feed_dict={X:batchImg,Y1:batchScale1,Y2:batchScale2,Y3:batchScale3})
            if(epoch%15 == 0):
                print(epochloss)
        imageNum = imageNum + BATCHSIZE
        Num = Num + 1
        if(Num%4 == 0):
            saver.save(sess,MODELPATH + 'MyModle__' + str(imageNum))            
        if(os.path.exists(STOPFLAGPATH)):
            saver.save(sess,MODELPATH + 'MyModle__Stop_' + str(imageNum))   
            print('checked stopfile,stop')
            break
return 0

And then I get some files:

MyModle__Stop_288.index
MyModle__Stop_288.meta
MyModle__Stop_288.data-00000-of-00001
checkpoint

Then I continue to train this model:

def reTrain():
with tf.Session() as sess:
    loder = tf.train.import_meta_graph('E:/MyYoloModel/MyModle__Stop_288.meta')
    loder.restore(sess, tf.train.latest_checkpoint('E:/MyYoloModel/'))
    graph = tf.get_default_graph()
    X = graph.get_tensor_by_name("X:0")
    Y1 = graph.get_tensor_by_name("Y1:0")
    Y2 = graph.get_tensor_by_name("Y2:0")
    Y3 = graph.get_tensor_by_name("Y3:0")
    Scale1 = graph.get_tensor_by_name("Scale1:0")
    Scale2 = graph.get_tensor_by_name("Scale2:0")
    Scale3 = graph.get_tensor_by_name("Scale3:0")  
    Loss = myYoloLoss([Scale1,Scale2,Scale3],[Y1, Y2 ,Y3])
    #error code 
    opt = tf.train.AdamOptimizer(2e-4).minimize(Loss)
    init = tf.global_variables_initializer()
    sess.run(init)
    batchImg,batchScale1,batchScale2,batchScale3 = getBatchImage(batchSize = BATCHSIZE)
    for epoch in range(10):
        _ ,epochloss = sess.run([opt,Loss],feed_dict={X:batchImg,Y1:batchScale1,Y2:batchScale2,Y3:batchScale3})
        print(epochloss)

And it will be occur this error: ValueError: Duplicate node name in graph: 'conv2d_0/kernel/Adam'
How can fix it?

Aloysia

Because you saved to file, I think you can just tf.reset_default_graph() after you finish training. tensorflow.org/api_docs/python/tf/reset_default_graph – Tews

Answer 1

I had a simmilar Error:

ValueError: Duplicate node name in graph: 'packed/0'

I think that this Error is due to a different Tensorfow Version than the code was programmed you are using. Try to downgrade the tf version while importin the package:

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

this trivial solutian was able to eliminate the problem

Staminody

Answer 2

The reason is that AdamOptimizer creates additional variables and operation in your graph. When you store your model, those operations are stored and loaded with the graph when you restore the model. If you run

tf.Graph.get_operations(graph)

you can see the list of operations that are loaded with you model. You will see operations that have /Adam or train/Adam init. When you try to find-tune or reuse you model, the new AdamOptimizer tries to create those operations again, hence it raises the "Duplicate node name" error. One way to fix the issue is to give a name to your new AdampOptimzer.

opt = tf.train.AdamOptimizer(2e-4m name='MyNewAdam').minimize(Loss)

However, We are not done yet. As you want to reuse the weight, you cannot initialize variable. However, if you will get error of uninitialized parameters when you run your training which is raised due to new AdamOptimizer variables which have not been initialized yet. To get around it, you need to initialize those new variables by :

uninitialized_vars = []
for var in tf.all_variables():
    try:
        sess.run(var)
    except tf.errors.FailedPreconditionError:
        uninitialized_vars.append(var)

tf.initialize_variables(uninitialized_vars)

Note: Unused nodes will not be executed and hence they won't affect training time.

Upstairs

Still got the same error. Do you have a more detailed code? I'm working with the tf.data as input pipeline. It's a little bit tricky. – Stricker

Moreover, in my case, with all the new node of Adam adding on without pruning the previous ones, the GPU memory might be explode. – Stricker

Answer 3

I also encountered this problem, and I solved it by changing some code in my virtual environment: venv\Lib\site-packages\tensorflow_core\python\framework\ops.py There is a method there called "unique_name". This method makes mistakes sometimes when it's building up names. Especially when it's using its name_stack. To remedy this I changed the method and added a small splitAndSet method which resolved the issue:

def splitAndSet(self, name, iInt):
#=================== Method splitAndSet =====================
    nameList = re.split("/", name)
    for nInt in range(len(nameList)):
        nameList[nInt] = "%s_%d" % (nameList[nInt], iInt)
    return "/".join(nameList)
#=============== END Method splitAndSet =====================

def unique_name(self, name, mark_as_used=True):
    if self._name_stack:
        name = self._name_stack + "/" + name

    # For the sake of checking for names in use, we treat names as case
    # insensitive (e.g. foo = Foo).
    name = self.splitAndSet(name, 0)
    name_key = self.splitAndSet(name.lower(), 0)
    i = self._names_in_use.get(name_key, 0)
    # Increment the number for "name_key".
    if mark_as_used:
        self._names_in_use[name_key] = i + 1
    if i > 0:
        base_name_key = name_key
        # Make sure the composed name key is not already used.
        while name_key in self._names_in_use:
            #name_key = "%s_%d" % (base_name_key, i)
            name_key = self.splitAndSet(base_name_key, i)
            i += 1
      # Mark the composed name_key as used in case someone wants
      # to call unique_name("name_1").
    if mark_as_used:
        self._names_in_use[name_key] = 1

    # Return the new name with the original capitalization of the given name.
    #name = "%s_%d" % (name, i - 1)
    name = self.splitAndSet(name, i - 1)
return name

Urinary

Answer 4

To give some context on how I got this error, I was trying to convert darknet weights to a TensorFlow model, following this tutorial. I got this error due to the following piece of code:

flags.DEFINE_string('output', './checkpoints/yolov4-416', 'path to output') 

I was giving the wrong path in the above code, it took me a lot of time to figure this out after trying so many Stackoverflow solutions. It's so trivial, but I hope it helps someone stuck on this error.

Lashley