Modify ResNet50 output layer for regression

Question

I am trying to create a ResNet50 model for a regression problem, with an output value ranging from -1 to 1.

I omitted the classes argument, and in my preprocessing step I resize my images to 224,224,3.

I try to create the model with

def create_resnet(load_pretrained=False):
  if load_pretrained:
        weights = 'imagenet'
  else:
      weights = None

  # Get base model
  base_model = ResNet50(weights=weights)

  optimizer = Adam(lr=1e-3)
  base_model.compile(loss='mse', optimizer=optimizer)

  return base_model

and then create the model, print the summary and use the fit_generator to train

   history = model.fit_generator(batch_generator(X_train, y_train, 100, 1),
                                  steps_per_epoch=300, 
                                  epochs=10,
                                  validation_data=batch_generator(X_valid, y_valid, 100, 0),
                                  validation_steps=200,
                                  verbose=1,
                                  shuffle = 1)

I get an error though that says

ValueError: Error when checking target: expected fc1000 to have shape (1000,) but got array with shape (1,)

Looking at the model summary, this makes sense, since the final Dense layer has an output shape of (None, 1000)

fc1000 (Dense)                  (None, 1000)         2049000     avg_pool[0][0]      

But I can't figure out how to modify the model. I've read through the Keras documentation and looked at several examples, but pretty much everything I see is for a classification model.

How can I modify the model so it is formatted properly for regression?

Harlot

Answer 1

Your code is throwing the error because you're using the original fully-connected top layer that was trained to classify images into one of 1000 classes. To make the network working, you need to replace this top layer with your own which should have the shape compatible with your dataset and task.

Here is a small snippet I was using to create an ImageNet pre-trained model for the regression task (face landmarks prediction) with Keras:

NUM_OF_LANDMARKS = 136

def create_model(input_shape, top='flatten'):
    if top not in ('flatten', 'avg', 'max'):
        raise ValueError('unexpected top layer type: %s' % top)

    # connects base model with new "head"
    BottleneckLayer = {
        'flatten': Flatten(),
        'avg': GlobalAvgPooling2D(),
        'max': GlobalMaxPooling2D()
    }[top]

    base = InceptionResNetV2(input_shape=input_shape,
                             include_top=False, 
                             weights='imagenet')

    x = BottleneckLayer(base.output)
    x = Dense(NUM_OF_LANDMARKS, activation='linear')(x)
    model = Model(inputs=base.inputs, outputs=x)
    return model

In your case, I guess you only need to replace InceptionResNetV2 with ResNet50. Essentially, you are creating a pre-trained model without top layers:

base = ResNet50(input_shape=input_shape, include_top=False)

And then attaching your custom layer on top of it:

x = Flatten()(base.output)
x = Dense(NUM_OF_LANDMARKS, activation='sigmoid')(x)
model = Model(inputs=base.inputs, outputs=x)

That's it.

You also can check this link from the Keras repository that shows how ResNet50 is constructed internally. I believe it will give you some insights about the functional API and layers replacement.

---

Also, I would say that both regression and classification tasks are not that different if we're talking about fine-tuning pre-trained ImageNet models. The type of task mostly depends on your loss function and the top layer's activation function. Otherwise, you still have a fully-connected layer with N outputs but they are interpreted in a different way.

Domiciliate