How to modify freezed layers in training using Tensorflow's Object Detection API?

Question

I am using Tensorflow's Object Detection API in training.

In which file, the freezed layers are defined to fine-tune the model in training. I need to experiment changing freezed layers in fine-tuning.

For example, if I use Resnet50 configuration, where I can change the freezed layers?

Tugman

If you freeze a layer, should it not be fine-tuned during training? – Nub

The title of your question is not very clear. A freezed layer is not supposed to be modified during training. Probably what you meant was How to freeze layers in training using TF OD API? – Nub

Answer 1

That certainly you can do.

By reading the proto file for training, there is a field called freeze_variables, this is supposed to be a list containing all variables that you want to freeze, e.g. excluding them during the training.

Supposed you want to freeze the weights from the first bottleneck in the first unit of the first block, you can do it by adding

freeze_variables: ["resnet_v1_50/block1/unit_1/bottleneck_v1/conv1/weights"]

so your config flie looks like this:

train_config: {
  batch_size: 1
  freeze_variables: ["resnet_v1_50/block1/unit_1/bottleneck_v1/conv1/weights"]
  ...

You can verify that the weights are in fact freezed by checking the tensorflow graph.

As shown, the weights do not have train operation anymore.

By choosing specific patterns for freeze_variables, you can freeze variables very flexibly (you can get layer names from the tensorflow graph).

Btw, here is the actual filtering operation.

Nub

Thanks for the good explanation. You said we can include in configuration file. But in this configuration file https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_resnet50_coco.config, there is no such freezed layer implementation. That is my query of where freezing is implemented in Tensorflow for faster_rcnn_resnet50. – Tugman

Oh, misunderstood that. But since the proto supports it, you can certainly add it even if the sample config does not contain it. – Nub

You mean Tensorflow Objection Detection API doesn't do transfer learning to only a few layers? It retrains all layers if there is no such freeze_variables: in train config file, is it? – Tugman

No, that is not what I meant. It can do transfer learning to any layers. If no freeze_variables, all layers' weight will be updated during training. – Nub

@danyfang How to know the possible values for freeze_variables? By default in train.proto it is written as: repeated string freeze_variables = 12;. What does it mean? – Makowski

@xanjay, I don't know exactly what this 12 means but I think it is just used for syntax. If u look at train.proto u will see that usually default values are provided with format [default = ...]. freeze_variables does not have default values. To find out the possible values u can do what I did by looking at the tensorflow graph. U can actually freeze any trainable variables, the only problem is to specify the exact variable name. – Nub

How can we see Tensorflow graph? I'm training mobilenetV1 but I don't know how can I see the name of different layers in it – Hoskins

Hi @danyfang, @Makowski At the time of this writing, the line repeated string freeze_variables = 12; is part of the TrainConfig Protocol Buffer message definition found in the proto file train.proto. The 12 is called a field number. A field number is a unique number that is assigned to each protobuff message’s field, and is used to identify during the serialising and deserializing process.You can read more about Protocol Buffers and field numbers at official docs – Serviette