How to convert a subset of numpy recarray to continuous array?
Asked Answered
U

1

2

I have a recarray that comes from reading a csv file. I am interested in converting a subset of columns to a continuous float array. I'd like to avoid converting them to list or stacking them one by one. I tried the suggestions in https://stackoverflow.com/a/11792956 and https://stackoverflow.com/a/7842620 but I get

ValueError: new type not compatible with array.

Here is my code:

a = np.recfromcsv(r"myfile.csv")
#a has many columns of type int, float or string. I want to extract those called coeff*
coeffs_columns = [n for n in a.dtype.names if n.startswith('coeff')] 
coeffs_recarray = a[coeffs_columns]
newtype=[(n,'<f8') for n in coeffs_columns]
b = coeffs_recarray.astype(newtype)
#b is:
#array((0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0),  dtype=[('coefficients00', '<f8'), ('coefficients1', '<f8'), ('coefficients2', '<f8'), ('coefficients3', '<f8'), ('coefficients4', '<f8'), ('coefficients5', '<f8'), ('coefficients6', '<f8'), ('coefficients7', '<f8'), ('coefficients8', '<f8'), ('coefficients9', '<f8'), ('coefficients100', '<f8'), ('coefficients11', '<f8'), ('coefficients12', '<f8'), ('coefficients13', '<f8'), ('coefficients14', '<f8')])
coeffs = b.view('<f8')

The "funny" thing is that if I extract only one column, or if I work with a recarray created as

x = np.array([(1.0, 2,7.0), (3.0, 4, 9.9)], 
                 dtype=[('x', '<f8'), ('y', '<f8'), ('z', '<f8')])

the conversion works.

Unchain answered 14/4, 2015 at 13:49 Comment(5)
Have you tried np.ascontiguousarray on your array of interest?Copperas
Does a[coeffs_columns].view('f') work?Tarragona
@Tarragona no, it gives the same resultUnchain
@Copperas , thanks, that has solved the problem! If you convert it to an answer I will accept itUnchain
another "funny" thing is that coeffs_recarray.flags['C_CONTIGUOUS'] is True , but it works only if I use explicitly np.ascontguousarrayUnchain
C
3

Numpy provides numpy.ascontiguousarray.

This function returns a contiguous array in memory (C order) of its input array. This is especially helpful when dealing with non contiguous views on arrays.

If Fortran order is desired, use numpy.asfortranarray.

Copperas answered 15/4, 2015 at 7:44 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.