Use pre-trained ResNet 50 in Keras for images with 4 channels

Question

Is there any way I can use the ImageNet weights for ResNet50 for my project which has images of shape (224,224,4)? The image has R,G,B,Y channels.

At the moment, I am simply using

 model = ResNet50(include_top=True, weights=None, input_tensor=None, input_shape=input_shape, pooling=None, classes=num_classes)

Now, if I need to use the ImageNet weights, I need to always set the number of classes to 1000. I tried doing that, and then popping the last layer, and adding my own Dense(num_classes) layer. However, now, the number of channels is an issue.

Can anyone suggest a way to accommodate 4 channels in the model while using the ImageNet weights?

Irairacund

No, there is not really a way, the network was just not trained with your extra channel. Your only option is to use RGB data. – Moser

Is there any way I can merge the channels? I used numpy.stack() to simply add the channels. However, I could use some advice on how I can accommodate 4 channels into 3. – Irairacund

That is something you have to experiment with, not something anyone can tell a propri. Remember that you and only you have this data, so just experiment and see what works best. – Moser

Kaggle protein challenge :)? Also, did you ever figure this out? – Distorted

Technically, you can fork keras.applications and modify network whatever you want. I've made a minimal working example of ResNet50 with imagenet weights (what's modified: 1st conv name: 'conv1_NEW', model.load_weights(weights_path, by_name=True)). Link: github.com/kohrah/rnd_stuff/blob/master/ResNet50_4channel.py – Height