Residual Neural Network: Concatenation or Element Addition?

Question

With the residual block in residual neural networks, is the addition at the end of the block true element addition or is it concatenation?

For example, would addition([1, 2], [3, 4]) produce [1, 2, 3, 4] or [4, 6] ?

Dullard

It is element-wise addition. – Calibre

Answer 1

It would result in [4, 6], and you can find out more in this paper

Merriott

Answer 2

The operation F + x is performed by a shortcut connection and element-wise addition

It is from the popular ResNet paper by Microsoft Research. Therefore it is element-wise addition, hence [4, 6]

Endemic