Tensorflow dataset with multiple inputs and target

Question

I am trying to implement a model with the ArcFace Layer: https://github.com/4uiiurz1/keras-arcface

to this extend I created a tf.data.dataset like so:

images= tf.data.Dataset.from_tensor_slices(train.A_image.to_numpy())
target = tf.keras.utils.to_categorical(
    train.Label.to_numpy(), num_classes=n_class, dtype='float32'
)
target = tf.data.Dataset.from_tensor_slices(target)

images= images.map(transform_img)

dataset = tf.data.Dataset.zip((images, target, target))

when I call model.fit(dataset)

I get the following error:

ValueError: Layer model expects 2 input(s), but it received 1 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=<unknown> dtype=float32>]

But this should work according:

tf.data with multiple inputs / outputs in Keras

Can someone point out my folly?

Thanks!

Edit: this solves some problems:

#reads in filepaths to images from dataframe train
images = tf.data.Dataset.from_tensor_slices(train.image.to_numpy())
#converts labels to one hot encoding vector
target = tf.keras.utils.to_categorical(train.Label.to_numpy(), num_classes=n_class, dtype='float32')
#reads in the image and resizes it
images= images.map(transform_img)
input_1 = tf.data.Dataset.zip((anchors, target))
dataset = tf.data.Dataset.zip((input_1, target))

And I think it's what we are trying. But I get a shape error for targets, it's (n_class, 1) instead of just (n_class,)

I.e. the fit methods throws this error

ValueError: Shapes (n_class, 1) and (n_class, n_class) are incompatible

and this warning

input expected is (None, n_class) but received an input of (n_class, 1)

Hooey

let say you have test data with no labels, how would you do it then. I have the same problem. for test data. my model takes 2 inputs and getting the same error as you.(expected 2 inputs got 1) – Asserted

Answer 1

I've made changes to the solution based on the arcface, you've wanted here is the code, i've managed to train it

The first one is from tensor slices as the original input and i used mnist to test it out

def map_data(inputs, outputs):
    image = tf.cast(inputs['image_input'], tf.float32)
    image = image / 255.
    image = tf.expand_dims(image, axis=2)
    
    labels = tf.one_hot(outputs, 10)
    
    return {'image_input': image, 'label_input': labels}, labels

dataset = tf.data.Dataset.from_tensor_slices(({
    'image_input': x_train, 'label_input': y_train
}, y_train))
dataset = dataset.map(map_data)
dataset = dataset.batch(2)

Here is the second type i have tried using a normal from tensor slices then i converted it to a multiple input, since both the normal labels are used for both the input and output

def map_data(images, annot_labels):
    image = tf.cast(images, tf.float32)
    image = image / 255.
    image = tf.expand_dims(image, axis=2) # convert to 0 - 1 range
    
    labels = tf.one_hot(annot_labels, 10)
    
    return {'image_input': image, 'label_input': labels}, labels

dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
dataset = dataset.map(map_data)
dataset = dataset.batch(2)

Dmz

Hi Edwin, nice to see you again, tried this too (actually in part as a consequence of our discussion yesterday but for a different model) But when I do this, it says it expected two inputs but only received 1 tensor – Hooey

Hmm i see, hold on let me try something – Dmz

This method actually throws the same shape error as my edit above: ....but it was called on an input with incompatible shape (1162, 1). edit: same warning but different error in the end – Hooey

Hi Olli, I have runned this with the arcface layer you tried to implement, this 1 is working according to my test – Dmz

Oh, thank you very much, I'll trust your word for it then and shamelessly copy-paste now. Thank you so much, I was wondering already if I may not have just reverted to generators. But I am so glad I'm learning tf.data.datasets now, it's considerably faster – Hooey

Btw i am using mnist for this i am not sure it fits your dataset, so do make changes accordingly – Dmz

Thanks, yes I'll adjust – Hooey

this worked beautifully I can't thank you enough, I've spent the better part of today working through this. Thank you!! If you're ever in zurich I'll buy you a beer – Hooey

Answer 2

I think you should do it like this:

target = tf.keras.utils.to_categorical(train.Label.to_numpy(), num_classes=n_class, dtype='float32')
    
images_target = tf.data.Dataset.from_tensor_slices((train.A_image.to_numpy(), target))

images_target = images_target.map(lambda x, y: (transform_img(x), y))
    

target = tf.data.Dataset.from_tensor_slices(target)
    
dataset = tf.data.Dataset.zip((images_target, target))

Angus

Thank you very much. I found that solution too, zipping the former and then the latter, at least almost the same. But then I get en error in the shape of target from the first zip, it adds an extra dimension in the end – Hooey

I also just tried your way with lambda (which is slightly different) get the following error: TypeError: <lambda>() missing 1 required positional argument: 'y' Is it possibly the result of target being one hot encoded? I.e. more than 1 input? – Hooey

maybe lambda x: (transform_img(x[0]), x[1]) works. Oh maybe it is the OE thing, not sure about that. – Angus

Could you edit your question with what you are trying? It's hard to read through comments. – Angus

Of course, thank you for your patience. I am still trying to solve the same thing, just made some progress, in line with what you suggested I think. I edited my question above to include that. – Hooey

Are you adding the batch size after that? something like dataset = dataset.batch(32), just to rule out some options. – Angus

Thanks, no, I did originally but now I put away everything and just do model.fit(dataset) – Hooey