Tensorflow - Value Error in model.fit - How to fix

Question

I am trying to train a Deep Neural Network using MNIST data set.

BATCH_SIZE = 100
train_data = train_data.batch(BATCH_SIZE)
validation_data = validation_data.batch(num_validation_samples)
test_data = scaled_test_data.batch(num_test_samples)

validation_inputs, validation_targets = next(iter(validation_data))

input_size = 784
output_size = 10
hidden_layer_size = 50

model = tf.keras.Sequential([
                    tf.keras.layers.Flatten(input_shape=(28,28,1)),
                    tf.keras.layers.Dense(hidden_layer_size, activation='relu'),
                    tf.keras.layers.Dense(hidden_layer_size, activation='relu'),
                    tf.keras.layers.Dense(output_size, activation='softmax')                        
                ])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

NUM_EPOCHS = 5
model.fit(train_data, epochs=NUM_EPOCHS, validation_data=(validation_inputs,validation_targets))

The model.fit is throwing the following error

-------------------------------------------------------------------------

--
ValueError                                Traceback (most recent call last)
<ipython-input-58-c083185dafc6> in <module>
      1 NUM_EPOCHS = 5
----> 2 model.fit(train_data, epochs=NUM_EPOCHS, validation_data=(validation_inputs,validation_targets))

~/anaconda3/envs/py3-TF2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
    726         max_queue_size=max_queue_size,
    727         workers=workers,
--> 728         use_multiprocessing=use_multiprocessing)
    729 
    730   def evaluate(self,

~/anaconda3/envs/py3-TF2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, **kwargs)
    222           validation_data=validation_data,
    223           validation_steps=validation_steps,
--> 224           distribution_strategy=strategy)
    225 
    226       total_samples = _get_total_number_of_samples(training_data_adapter)

~/anaconda3/envs/py3-TF2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py in _process_training_inputs(model, x, y, batch_size, epochs, sample_weights, class_weights, steps_per_epoch, validation_split, validation_data, validation_steps, shuffle, distribution_strategy, max_queue_size, workers, use_multiprocessing)
    562                                     class_weights=class_weights,
    563                                     steps=validation_steps,
--> 564                                     distribution_strategy=distribution_strategy)
    565     elif validation_steps:
    566       raise ValueError('`validation_steps` should not be specified if '

~/anaconda3/envs/py3-TF2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py in _process_inputs(model, x, y, batch_size, epochs, sample_weights, class_weights, shuffle, steps, distribution_strategy, max_queue_size, workers, use_multiprocessing)
    604       max_queue_size=max_queue_size,
    605       workers=workers,
--> 606       use_multiprocessing=use_multiprocessing)
    607   # As a fallback for the data type that does not work with
    608   # _standardize_user_data, use the _prepare_model_with_inputs.

~/anaconda3/envs/py3-TF2/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/data_adapter.py in __init__(self, x, y, sample_weights, batch_size, epochs, steps, shuffle, **kwargs)
    252     if not batch_size:
    253       raise ValueError(
--> 254           "`batch_size` or `steps` is required for `Tensor` or `NumPy`"
    255           " input data.")
    256 

ValueError: `batch_size` or `steps` is required for `Tensor` or `NumPy` input data.

The training and validation data are obtained from MNIST dataset. Some part of the data are taken as training data and some as testing data.

What am I doing wrong here?

Update As per Dominques suggestion, I have changed model.fit to

model.fit(train_data, batch_size=128, epochs=NUM_EPOCHS, validation_data=(validation_inputs,validation_targets))

But now, I get the following error

ValueError: The `batch_size` argument must not be specified for the given input type. Received input: <BatchDataset shapes: ((None, 28, 28, 1), (None,)), types: (tf.float32, tf.int64)>, batch_size: 128

Curler

What does train_data = train_data.batch(BATCH_SIZE) return? One batch? An iterator for batches? Try feeding a simple tuple numpy array's of the form (X_train, y_train) as well as batch_size argument and you should be fine. – Foil

As for your new error with batch_size, here is the reason------batch_size: Integer or None. Number of samples per gradient update. If unspecified, batch_size will default to 32. Do not specify the batch_size if your data is in the form of symbolic tensors, datasets, generators, or keras.utils.Sequence instances (since they generate batches). – Quinidine

i was also facing exact issue, by adding validation_steps parameter it got resolved. the description says that if it is not specified it will continue till the validation data gets exhausted, but doesn't seem to work. instead validation_steps works thanks – Banking

Answer 1

The tf doc will give you more clues why you get the error.

https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit

validation_data: Data on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. validation_data will override validation_split. validation_data could be:
    •   tuple (x_val, y_val) of Numpy arrays or tensors
    •   tuple (x_val, y_val, val_sample_weights) of Numpy arrays
    •   dataset 

For the first two cases, batch_size must be provided. For the last case, validation_steps must be provided.

Since You already have the validation dataset batched, consider to use it directly and specify validation steps as below.

BATCH_SIZE = 100
train_data = train_data.batch(BATCH_SIZE)
validation_data = validation_data.batch(BATCH_SIZE)
...
model.fit(train_data, epochs=NUM_EPOCHS, validation_data=validation_data,validation_steps=1)

Quinidine

Thank you for your help. I am new to tensorflow and trying to learn. When I am not batching validation data set , I am getting error regarding the shape of validation dataset. Could you please explain in a code snippet? – Curler

Why validation_steps=1 instead of other number? – Disjoint

@Disjoint it is just an example, validation_steps is the number of steps(batches of samples) to evaluate model against the dataset. Any number smaller than the total number of batches of the evaluation dataset should be okay. – Quinidine

Answer 2

You need to specify the batch size, i.e. how many data points should be included in each iteration. If you look at the documentation you will see that there is no default value set.

https://www.tensorflow.org/api_docs/python/tf/keras/Sequential

you can set the value by adding batch_size to the fit command. Good values are normally numbers along the line of 2**n, as this allows for more efficient processing with multiple cores. For you this shouldn't make a strong difference though :)

model.fit(train_data, 
          batch_size=128
          epochs=NUM_EPOCHS, 
          validation_data=(validation_inputs,validation_targets))

Subjoin

Answer 3

Why nobody mention i don't know but your problem is Y_train data. You don't supply it as an argument to your model..

model.fit(X_Train, y_train, batch_size=None, epochs=1, verbose=1, callbacks=None, validation_split=0.0, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0, steps_per_epoch=None, validation_steps=None, validation_freq=1, max_queue_size=10, workers=1, use_multiprocessing=False)

Instead of y_train you are giving :

model.fit(train_data, batch_size=128 ....

And getting an Error saying :

ValueError: `batch_size` or `steps` is required for `Tensor` or `NumPy` input data.

I hope it helps.

Virtuosity

Please post an answer, if you are clear about the question/error. It seems that train_data is generator, whose output is list of input and output. Please follow the guide while answering stackoverflow.com/help/how-to-answer – Muire

New users please read the code of conduct before making post or answer? – Locklin

@ankish-bansal , and santosh-aryal / I am very confident about my answer ,It is my first msg and you are trying to discourage me impolitely.I am just new here not at Data Science. Just look the link : It is the same MNIST data set with Keras : keras.io/examples/mnist_cnn . In addition generator should be something like that : ('tr_dataset = tf.data.Dataset.from_tensor_slices((tr_data, tr_label))') where do you see it in his code? he has only : ('train_data = train_data.batch(BATCH_SIZE)') , where is his Y_training ? . Please don't discourage people to share info – Virtuosity

No no its not about the discouraging new comers, many new users are making a post or answering but they are getting down votes because of their post. Therefore I was suggesting to write a clear solutions so that the solutions too will help others. However, there is a accepted answer of this post, so I was suggesting you to make an explanation of your answer. Its great of your confidence. – Locklin

@santosh-aryal ok thank you for taking time and explanation ..Sorry if I was excited answering to you too..But you know I dont think so it was a stupid answer..I was already writing a project in Keras (for me) and did see he is suffering something similar, and shared my idea.And as I said not beginner in Data Science, but of course saying expert far away too..Any way thank you. – Virtuosity

@everytu4 StackOverflow has a woderful community. We are just helping you, to feel more comfortable and confident, while answering on this platform. I wrote, because of your statement Why nobody mention i don't know. Please don't get discourage of anything and continue. – Muire

Answer 4

model.fit(train_data, epochs=NUM_EPOCHS, validation_data=(validation_inputs, validation_targets), verbose=2) 

change to (by adding validation_steps=1) will do the trick

model.fit(train_data, epochs=NUM_EPOCHS, validation_data=(validation_inputs, validation_targets),validation_steps=1, verbose=2)

Reentry

Answer 5

I changed the input_shape=(28,28,1) to input_shape=(28,28,3) and it worked for me.

Traverse