Understanding NumPy's Convolve

Question

When calculating a simple moving average, numpy.convolve appears to do the job.

Question: How is the calculation done when you use np.convolve(values, weights, 'valid')?

When the docs mentioned convolution product is only given for points where the signals overlap completely, what are the 2 signals referring to?

If any explanations can include examples and illustrations, it will be extremely useful.

window = 10
weights = np.repeat(1.0, window)/window
smas = np.convolve(values, weights, 'valid')

Capsulate

numpy.convolve from the NumPy Manual and the references at the bottom explain convolution and this specific implementation very well. – Unconformity

Answer 1

Convolution is a mathematical operator primarily used in signal processing. Numpy simply uses this signal processing nomenclature to define it, hence the "signal" references. An array in numpy is a signal. The convolution of two signals is defined as the integral of the first signal, reversed, sweeping over ("convolved onto") the second signal and multiplied (with the scalar product) at each position of overlapping vectors. The first signal is often called the kernel, especially when it is a 2-D matrix in image processing or neural networks, and the reversal becomes a mirroring in 2-D (NOT transpose). It can more clearly be understood using the animations on wikipedia.

Convolutions have multiple definitions depending on the context. Some start the convolution when the overlap begins while others start when the overlap is only partial. In the case of numpy's "valid" mode, the overlap is specified to be always complete. It is called "valid" since every value given in the result is done without data extrapolation.

For instance, if your array X have a length of 2 and your array Y have a length of 4, the convolution of X onto Y in "valid" mode will give you an array of length 3.

First step, for X = [4 3] and Y = [1 1 5 5]:

[3 4]                   (X is reversed from [4 3] to [3 4], see note)
[1 1 5 5]
= 3 * 1 + 4 * 1 = 7

Note: If X was not reversed, the operation would be called a cross-correlation instead of a convolution.

Second Step:

  [3 4]
[1 1 5 5]
= 3 * 1 + 4 * 5 = 23

Third step:

    [3 4]
[1 1 5 5]
= 3 * 5 + 4 * 5 = 35

The result of the convolution for mode "valid" would then be [7 23 35].

If the overlap is be specified as one single data point (as the case in mode "full"), the result would have given you an array of length 5. The first step being:

[3 4]
  [1 1 5 5]
= 3 * undefined (extrapolated as 0) + 4 * 1 = 4

And so on. More extrapolation modes exist.

Sausauce

Just to clarify things up: the vector [2 3] is mirrored to be [3 2] in the explained convolution. This is correct. If the vector was not mirrored, the operation would be called a cross-correlation. This is due to the mathematical definition of the convolution. – Sausauce

What about the same mode in convolution? – Ballon

@RamonMartinez This mode is described in the numpy documentation. Basically, it will perform just as explained for the "full" mode here, but the results will be truncated (at the left and right equally) to be the same length as the longest input. – Sausauce

Not sure why this has not been accepted as the correct answer yet. – Parasite

Thanks so much! Very helpful to a person who just understood 2D convolution from convolutional neural network but no idea what 1-D convolution does. – Taneka

I tried this in numpy np.convolve([3, 4], [1, 1, 5, 5], 'valid') the output is array([ 7, 19, 35]) I think the Second Step is wrong in my case, though I have no idea how and why. Any respone will be grateful. – Caralie

The example is for [4,3] as first parameter to np.convolve(), not [3,4]. By mathematical definition, convolution reverses the first parameter. – Sausauce

@Soravux: You should add the note that the first parameter is reversed. – Cicada

Answer 2

It is notable also that the kernel is "centered" in the sense that indices for the kernel are taken with respect to the centre element of the array. In other words, for arrays with index starting at 0 (as in python), the function B = np.convolve (A, K) computes

where m = (len(K) - 1)//2 (integer division). This is an integer, also when len(K) is even.

The summation is nominally over all values of i from -∞ to ∞, where values of A out of range are assumed equal to zero. The same is true for values of the kernel. For np.convolution2D, you have to option of using the mode, boundary and fillvalue options to specify how values of A out of range are treated.

Thus, for example, you get different answers for np.convolve(A, K) if K = np.array([1, 2, 3]) or K = np.array([1, 2, 3, 0, 0])

Constitutional