How does import_array in numpy C API work?

Question

I am trying to convert a c-style array in c++ to a numpy array and ran into problems when trying to use the "PyArray_SimpleNewFromData" function. It turns out I need to call

import_array()

Though I do not understand how to call this function. Whenever I try calling it I get compiler error which I do not manage to understand. For instance writing the following simple script:

#include <Python.h>
#include <numpy/arrayobject.h>

int main(){
    Py_Initialize();
    import_array();
    Py_Finalize();
    return 0;
}

produces the compiler error

error: return-statement with no value, in function returning 'int' [-fpermissive] import_array();

I looked at several examples, such as :

Numpy C-Api example gives a SegFault

PyArray_SimpleNewFromData example

https://codereview.stackexchange.com/questions/92266/sending-a-c-array-to-python-numpy-and-back

Numpy/CAPI error with import_array() when compiling multiple modules

But whatever I try (even when seemingly following those examples) I seem to run into the compiler error above. What am I missing or doing wrong? An explanation on how the import_array() function should be called would be very welcome. Thanks for the help!

Update:

I am using python 2.7.11, and I think it might be related to what is discussed here :

https://github.com/clemenscorny/brisk/issues/3

but I still have no idea how to fix it.

Lvov

Answer 1

Reading the source code of import_array() from numpy, it is a macro that mainly calls the _import_array() regular function, and then does some weird exception handling. If you call _import_array() instead of import_array(), the compilation error disappears. And, at least in my case, the behavior is then correct (i.e. no segfault when calling Numpy C API functions)

Nansen

FWIW, the "weird" exception handling is just the standard way of setting an exception for CPython. It would only be relevant relevance if you will eventually return to the Python interpreter (with an exception set). – Xena

Answer 2

It's a bug in numpy: import_array() is a macro that can return 0 or NULL or nothing (void).

It's impossible to use in strict mode where the compiler verifies the return type of functions. A function returning an integer can't return NULL. A function returning a pointer can't return 0. A function returning something can't return nothing.

There's an alternative macro to use instead import_array1(0)

int
numpy_init()
{
    import_array1(0);
    return 0;
}

See numpy source code __multiarray_api.h

#if PY_VERSION_HEX >= 0x03000000
#define NUMPY_IMPORT_ARRAY_RETVAL NULL
#else
#define NUMPY_IMPORT_ARRAY_RETVAL
#endif

#define import_array() {if (_import_array() < 0) {PyErr_Print(); PyErr_SetString(PyExc_ImportError, "numpy.core.multiarray failed to import"); return NUMPY_IMPORT_ARRAY_RETVAL; } }

#define import_array1(ret) {if (_import_array() < 0) {PyErr_Print(); PyErr_SetString(PyExc_ImportError, "numpy.core.multiarray failed to import"); return ret; } }

#define import_array2(msg, ret) {if (_import_array() < 0) {PyErr_Print(); PyErr_SetString(PyExc_ImportError, msg); return ret; } }

#endif

Countryfied

This must be an accepted answer. – Alveraalverez

This isn't really a bug in numpy; import_array() is meant for use in a CPython extension function where the function would normally return PyObject *. You probably want import_array1(-1) (return some kind of error indication on failure - not zero), or just plain _import_array() and do your own error handling. – Xena