What is Adaptive average pooling and How does it work?

Question

I recently came across a method in Pytorch when I try to implement AlexNet. I don't understand how it works. Please explain the idea behind it with some examples. And how it is different from Maxpooling or Average poling in terms of Neural Network functionality

nn.AdaptiveAvgPool2d((6, 6))

Diphtheria

Answer 1

In average-pooling or max-pooling, you essentially set the stride and kernel-size by your own, setting them as hyper-parameters. You will have to re-configure them if you happen to change your input size.

In Adaptive Pooling on the other hand, we specify the output size instead. And the stride and kernel-size are automatically selected to adapt to the needs. The following equations are used to calculate the value in the source code.

Stride = (input_size//output_size)  
Kernel size = input_size - (output_size-1)*stride  
Padding = 0

Aquatic

How come Stride = (input_size//output_size)? – Guillen

Found another answer that explains this formula is only the case when the input size is an integer multiple of the second dimension. Otherwise, the flooring that occurs when calculating the output dimensions will cause this formula to be invalid. – Guillen

Thank you very much. Where did you get these formulas from please? – Bonanza

I'm not familiar with how Adaptive Pooling is implemented in specific libraries, but wanted to note that in the case of input_size being a multiple of output_size, if input_size//output_size is substituted for stride in the expression for Kernel size above, then we obtain Kernel size = stride. Actually, it seems to me that the formula for Kernel size above is equivalent to Kernel size = stride + (input_size % output_size). Maybe this helps in understanding the intent for Kernel size's expression in the answer? – Wilding

Answer 2

Here is a discussion about this problem:

import torch
from torch import nn
a = torch.arange(0,14., requires_grad=True).view(1,1,1,-1)
m = nn.AdaptiveAvgPool2d((None, 4))
b = m(a)
print(b)

The output is:

tensor([[[[ 1.5000,  4.5000,  8.5000, 11.5000]]]],
       grad_fn=<AdaptiveAvgPool2DBackward0>)

This four numbers are the mean(a[0:4]), mean(a[3:7]), mean(a[7:11]) and mean(a[10:14]), so there is not a constant kernel size and stride.

Nutrition