element wise sum of structured arrays in numpy

Question

I wonder if it is possible at all to perform element-wise sum (or other operations) of two structured numpy arrays of an identical shape.

arr1 = np.array([[1,2,3],[2,3,4]], dtype=[("x", "f8"),("y", "f8")])
arr2 = np.array([[5,4,3],[9,6,4]], dtype=[("x", "f8"),("y", "f8")])
arr3 = np.sum(arr1, arr2)

says "ufunc 'add' did not contain a loop with signature matching types dtype([('x', '

If it is not possible, it will be great to understand why that is impossible or impractical to implement in numpy.

Oenomel

Have to do the sum, or other math field by field. In general fields may include strings and/or other dtypes that don't implement math. – Dehydrogenate

if the dtype is uniform... arr.view((arr.dtype[0], len(arr.dtype.names))).sum(axis=0) … but separate it out into components so you can see what is going on – Herwig

@Dehydrogenate That's very true. I never store strings in ndarrays, so I overlooked that simple fact...! If you want, I will take it as the answer. It may look too short, but your comment clearly solved the problem :) – Oenomel

This works for you? arr.view(np.float32).reshape(arr.shape + (-1,)) I think it is similar to @Herwig solution – Domestic

@Herwig - I think my example was confusing. I've updated the example. Your previous suggestion is only applicable to a single array and shrinks the array along an axis. So, I'm afraid they don't work. – Oenomel

@JonnyCrunch - I've updated the example. Like NaN's suggestion, your previous suggestion is only applicable to a single array and it duplicates the array. I must have confused you. – Oenomel

related : #50931921 – Kloof

Normally structured arrays are initialized with list(s) of tuples. Since you don't have tuples it creates a (2,2) array with the x and y values repeated. @JonnyCrunch s point is that in some cases a structured array can be viewed as a uniform simple dtype, but you have to watch the shape. – Dehydrogenate

Expanding on the view idea, it is possible to define a dtype with overlapping fields. In this case a dtype that defines both x and y, and an xy field that occupies the same slots. IN that case you could do math on arr['xy'] and see the results in the x,y fields. But we'd have to study the dtype documentation to do that right. – Dehydrogenate

Answer 1

With your array:

In [236]: arr1 = np.array([[1,2,3],[2,3,4]], dtype=[("x", "f8"),("y", "f8")])
In [237]: arr1
Out[237]: 
array([[(1., 1.), (2., 2.), (3., 3.)],
       [(2., 2.), (3., 3.), (4., 4.)]], dtype=[('x', '<f8'), ('y', '<f8')])
In [238]: arr1['x']
Out[238]: 
array([[1., 2., 3.],
       [2., 3., 4.]])

Normally the data for a structured array is provided in the form a list(s) of tuples., same as displayed in Out[237]. Without the tuples np.array assigns the same value to both fields.

You have to do math on each field separately:

In [239]: arr1['y'] *= 10
In [240]: arr1
Out[240]: 
array([[(1., 10.), (2., 20.), (3., 30.)],
       [(2., 20.), (3., 30.), (4., 40.)]],
      dtype=[('x', '<f8'), ('y', '<f8')])

Math operations are defined for simple dtypes like int and float, and uses compiled code where possible.

This error means that the add ufunc has not been defined for this compound dtype. And I think that's true for all compound dtypes.

In [242]: arr1 + arr1
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-242-345397c600ce> in <module>()
----> 1 arr1 + arr1

TypeError: ufunc 'add' did not contain a loop with signature matching types dtype([('x', '<f8'), ('y', '<f8')]) dtype([('x', '<f8'), ('y', '<f8')]) dtype([('x', '<f8'), ('y', '<f8')])

Since the fields in this case have the same base dtype, we can define another compound dtype that can 'view' it:

In [243]: dt2 = np.dtype([('xy', 'f8', 2)])
In [244]: arr2 = arr1.view(dt2)
In [245]: arr2
Out[245]: 
array([[([ 1., 10.],), ([ 2., 20.],), ([ 3., 30.],)],
       [([ 2., 20.],), ([ 3., 30.],), ([ 4., 40.],)]],
      dtype=[('xy', '<f8', (2,))])
In [246]: arr2['xy']
Out[246]: 
array([[[ 1., 10.],
        [ 2., 20.],
        [ 3., 30.]],

       [[ 2., 20.],
        [ 3., 30.],
        [ 4., 40.]]])

Math on that field will be seen in the original array:

In [247]: arr2['xy'] += .1
In [248]: arr2
Out[248]: 
array([[([ 1.1, 10.1],), ([ 2.1, 20.1],), ([ 3.1, 30.1],)],
       [([ 2.1, 20.1],), ([ 3.1, 30.1],), ([ 4.1, 40.1],)]],
      dtype=[('xy', '<f8', (2,))])
In [249]: arr1
Out[249]: 
array([[(1.1, 10.1), (2.1, 20.1), (3.1, 30.1)],
       [(2.1, 20.1), (3.1, 30.1), (4.1, 40.1)]],
      dtype=[('x', '<f8'), ('y', '<f8')])

We can also view it as a simple dtype, but will have to adjust the shape:

In [250]: arr3 = arr1.view('f8')
In [251]: arr3
Out[251]: 
array([[ 1.1, 10.1,  2.1, 20.1,  3.1, 30.1],
       [ 2.1, 20.1,  3.1, 30.1,  4.1, 40.1]])
In [252]: arr3.reshape(2,3,2)
Out[252]: 
array([[[ 1.1, 10.1],
        [ 2.1, 20.1],
        [ 3.1, 30.1]],

       [[ 2.1, 20.1],
        [ 3.1, 30.1],
        [ 4.1, 40.1]]])

Dehydrogenate

I like the trick using a view to get a "pure" number array! I think this is overkill to the present problem, but will be useful at some point. – Oenomel