Is there any way to make a soft reference or Pointer-like objects using Numpy arrays?

Question

I was wondering whether there is a way to refer data from many different arrays to one array, but without copying it.

Example:

import numpy as np
a = np.array([2,3,4,5,6])
b = np.array([5,6,7,8])

c = np.ndarray([len(a)+len(b)])

offset = 0
c[offset:offset+len(a)] = a
offset += len(a)
c[offset:offset+len(b)] = b

However, in the example above, c is a new array, so that if you modify some element of a or b, it is not modified in c at all.

I would like that each index of c (i.e. c[0], c[1], etc.) refer to each element of both a and b, but like a pointer, without making a deepcopy of the data.

Ilan

No can do... You can do it the other way around, i.e. generate slices a and b from a previously defined c, so with some previous planning you may get what you want. But not after the fact. – Witcher

This is interested indeed, but I have a and b "by the self definition of my problem". Thanks – Ilan

Answer 1

As @Jaime says, you can't generate a new array whose contents point to elements in multiple existing arrays, but you can do the opposite:

import numpy as np

c = np.arange(2, 9)
a = c[:5]
b = c[3:]
print(a, b, c)
# (array([2, 3, 4, 5, 6]), array([5, 6, 7, 8]), array([2, 3, 4, 5, 6, 7, 8]))

b[0] = -1

print(c,)
# (array([ 2,  3,  4, -1,  6,  7,  8]),)

---

I think the fundamental problem with what you're asking for is that numpy arrays must be backed by a continuous block of memory that can be regularly strided in order to map memory addresses to the individual array elements.

In your example, a and b will be allocated within non-adjacent blocks of memory, so there will be no way to address their elements using a single set of strides.

Zo

Thanks for your answer, but it seems to me that there must be a way of defining an array of pointers. It should be very useful to know how to define pointers in Python. Is there any way to do that? – Ilan

That depends on what exactly you mean by "an array of pointers". As I already explained, the internal memory layout of numpy arrays means that you definitely can't create a new numpy array that references values in two existing arrays that were allocated separately. I think this might be an XY problem - what exactly do you want an "array of pointers" for? – Zo

With an array of pointers I mean an array such that each element points to another memory address. But I think you are right, it is not possible. Anyways, you example was very intereseting indeed. Thanks! – Ilan

My question was really about why you wanted such an array. I see you accepted my answer anyway, but if you wanted to ask a separate follow-up question then I'd strongly suggest you explain the background of your problem more fully - there's probably a good way to solve it with numpy, but we need to know what your actual problem is, rather than just the particular solution you had in mind. – Zo