How to Fine tune existing Tensorflow Object Detection model to recognize additional classes? [closed]

Question

Thanks to Google for providing a few pre-trained models with tensorflow API.

I would like to know how to retrain a pre-trained model available from the above repository, by adding new classes to the model. For example, the trained COCO dataset model has 90 classes, I would like to add 1 or 2 classes to the existing one and get one 92 class object detection model as a result.

Running Locally is provided by the repository but it is completely replacing those pre-trained classes with newly trained classes. There, only train and eval are mentioned.

So, is there any other way to retrain the model and get 92 classes as a result?

Rhearheba

This is a hot research topic, let me know if you'll find a nice solution. As a quick and dirty hack: to the layer before last 90-class FC layer attach a 2-class FC layer. Train only that tiny layer, so that 90 classes are intact. On test time run both layers and concat the results into 92 classes. Will this work? If yes, write a paper and add me as a coauthor ;) – Graycegrayheaded

This might help: #47592250 – Mukund

I’m voting to close this question because this is not a software development question and would be more suited to a different Stack Exchange site such as Cross Validated or Data Science – Cholula

Answer 1

Question : How do we add a few more classes to my already trained network?

Specifically, we want to keep all the network as-is other than the output of the new classes. This means that for something like ResNet, we want to keep everything other than the last layer frozen, and somehow expand the last layer to have our new classes.

Answer : Combine the existing last layer with a new one you train

Specifically, we will replace the last layer with a fully connected layer that is large enough for your new classes and the old ones. Initialize it with random weights and then train it on your classes and just a few of the others. After training, copy the original weights of the original last fully connected layer into your new trained fully connected layer.

If, for example, the previous last layer was a 1024x90 matrix, and your new last layer is a 1024x92 matrix, copy the 1024x90 into the corresponding space in your new 1024x92. This will destructively replace all your training of the old classes with the pre-trained values but leave your training of your new classes. That is good, because you probably didn't train it with the same number of old classes. Do the same thing with the bias, if any.

Your final network will have only 1024x2 new weight values (plus any bias), corresponding to your new classes.

A word of caution, although this will train fast and provide quick results, it will not perform as well as retraining on a full and comprehensive data set.

That said, it'll still work well ;)

Here is a reference to how to replace the last layer How to remove the last layer from trained model in Tensorflow that someone else answered

Unbelt