Behavior of Dropout layers in test / training phase

Question

According to the Keras documentation dropout layers show different behaviors in training and test phase:

Note that if your model has a different behavior in training and testing phase (e.g. if it uses Dropout, BatchNormalization, etc.), you will need to pass the learning phase flag to your function:

Unfortunately, nobody talks about the actual differences. Why should dropout behave differently in test phase? I expect the layer to set a certain amount of neurons to 0. Why should this behavior depend on the training/test phase?

Alimentation

Answer 1

Dropout is used in the training phase to reduce the chance of overfitting. As you mention this layer deactivates certain neurons. The model will become more insensitive to weights of other nodes. Basically with the dropout layer the trained model will be the average of many thinned models. Check a more detailed explanation here

However, in when you apply your trained model you want to use the full power of the model. You want to use all neurons in the trained (average) network to get the highest accuracy.

Sigismond

This means that dropout is disabled in test phase? – Alimentation

exactly, it is disabled automatically – Sigismond

Any update on that? Is there a better way to alleviate that switch? – Reprography

Check the answers here for implementing a permanent dropout layer. Which also is enabled in test phase. #47787511 – Sigismond