Tensorflow/Keras Conv2D layers with padding='SAME' behave strangely

Question

My question:

A straightforward experiment that I conducted showed that using padding='SAME' in a conv2d layer in Keras/TF is different from using padding='VALID' with a preceding zero-padding layer.

1. How is that possible?
2. Does Keras/TF pads zeros symmetrically around the tensor?

Explanation of the experiment - just if you're interested in reading further:

I used the onnx2keras package to convert my Pytorch model into keras/TF.

When onnx2keras encounters a convolutional layer with padding > 0 in the ONNX model, it translates it to Keras' Conv2D with valid padding (i.e., no padding!), preceded by Keras' ZeroPadding2D layer. This works very well and returns outputs that are identical to those produced by the Pytorch network.

I yet thought it was strange that it didn't simply used padding='SAME', as most of the references say that Keras/TF use zero padding, just like Pytorch does.

Nevertheless, I patched onnx2keras and made it produce me Conv2D layers with padding='SAME' rather than the existing solution of 'VALID' padding with a preceding zero-padding layer. This made the resulting model return different outputs than the one with the zero-padding layer, and of course different from my Pytorch model, which was identical until the patch.

Herve

as per the documentation "valid" is no padding while "same" results in padding the input such that the output has the same length as the original input. – Dillard

@nickthefreak of course. But I expected "same" to work the same way as when manually adding a padding layer before the convolution and then using "valid", but this is not what happens – Herve

@Herve - It will be clear to explain or identify the error if you post the simple reproducible code that you experimented. – Neolithic

Answer 1

padding='Same' in Keras means padding is added as required to make up for overlaps when the input size and kernel size do not perfectly fit.

Example of padding='Same':

# Importing dependency
import keras
from keras.models import Sequential
from keras.layers import Conv2D

# Create a sequential model
model = Sequential()

# Convolutional Layer
model.add(Conv2D(filters=24, input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2) ,padding='Same'))

# Model Summary
model.summary()

Output of the code -

Model: "sequential_20"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_28 (Conv2D)           (None, 3, 3, 24)          120       
=================================================================
Total params: 120
Trainable params: 120
Non-trainable params: 0
_________________________________________________________________

Pictorial Representation: Below image shows how the padding for the input (input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2)) when padding='Same'.

------------------------------------------------------------------------------------------------------------------

padding='Valid' in Keras means no padding is added.

Example of padding='Valid': Have used same input for Conv2D that we used above for padding = 'Same' .i.e. (input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2))

# Importing dependency
import keras
from keras.models import Sequential
from keras.layers import Conv2D

# Create a sequential model
model = Sequential()

# Convolutional Layer
model.add(Conv2D(filters=24, input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2) ,padding='Valid'))

# Model Summary
model.summary()

Output of the code -

Model: "sequential_21"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_29 (Conv2D)           (None, 2, 2, 24)          120       
=================================================================
Total params: 120
Trainable params: 120
Non-trainable params: 0
_________________________________________________________________

Pictorial Representation: Below image shows there is no padding added for the input (input_shape=(5,5,1), kernel_size=(2,2), strides =(2,2)) when padding='Valid'.

------------------------------------------------------------------------------------------------------------------

Now lets try same code that we used for padding='Valid' for the input (input_shape=(6,6,1), kernel_size=(2,2), strides =(2,2)). Here padding='Valid' should behave same as padding='Same'.

Code -

# Importing dependency
import keras
from keras.models import Sequential
from keras.layers import Conv2D

# Create a sequential model
model = Sequential()

# Convolutional Layer
model.add(Conv2D(filters=24, input_shape=(6,6,1), kernel_size=(2,2), strides =(2,2) ,padding='Valid'))

# Model Summary
model.summary()

Output of the code -

Model: "sequential_22"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_30 (Conv2D)           (None, 3, 3, 24)          120       
=================================================================
Total params: 120
Trainable params: 120
Non-trainable params: 0
_________________________________________________________________

Neolithic

Thank you so much for the very detailed answer. So if I understood correctly from the first figure, the padding in TF is done by duplicating the right and bottom vectors? – Herve

Yes. You are correct. The padding='Same' results in padding the input such that the kernel covers all the pixel for the input_shape=(5,5,1) with Stride=(2,2). If Stride=(1,1) then, padding='Same' results in padding the input such that the output has the same length as the original input. – Neolithic

Good to know. This is in contradiction to PyTorch, which pads with zeros from all the sides. This is the reason why onnx2keras translates a conv2d layer with padding into two different layers in TF - first a zero padding layer and then a "valid" conv2d layer – Herve