Cython Numpy warning about NPY_NO_DEPRECATED_API when using MemoryView

Question

I am converting a Cython memoryview to a numpy array (to be able to use it in pure Python code):

from libc.stdlib cimport realloc
cimport numpy as np

DTYPE = np.float64
ctypedef np.float64_t DTYPE_t

cpdef np.ndarray[DTYPE_t] compute(DTYPE_t[:,::1] data):
    cdef unsigned int Nchannels = data.shape[0]
    cdef unsigned int Ndata = data.shape[1]
    cdef DTYPE_t* output = NULL
    cdef DTYPE_t[::1] mv

    output = <DTYPE_t*>realloc(output, Ndata*sizeof(output))
    if not output:
        raise MemoryError()
    mv = <DTYPE_t[:Ndata]>output
    mv[10:Ndata-10] = 0.0
    # various calculations...
    return np.asarray(mv, dtype=DTYPE, order='C')

It compiles, but the compiler gives the following warning:

/Users/vlad/anaconda/lib/python2.7/site-packages/numpy/core/include
/nump/npy_1_7_deprecated_api.h:15:2: warning:
"Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]

I added the suggested directive in setup.py:

from distutils.core import setup, Extension
from Cython.Build import cythonize
import numpy

filename = 'agents3.pyx'

agents_module = Extension(
    'Agents',
    sources = [filename],
    define_macros = [('NPY_NO_DEPRECATED_API', 'NPY_1_7_API_VERSION')],
    include_dirs = [numpy.get_include()],
)

setup (name = 'Agents',
    ext_modules = cythonize(agents_module)
)

Now it wouldn't compile, it says:

Vlads-MacBook-Pro:program vlad$ python setup.py build_ext --inplace
Compiling agents3.pyx because it changed.
Cythonizing agents3.pyx
running build_ext
building 'Agents' extension
gcc -fno-strict-aliasing -I/Users/vlad/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/Users/vlad/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/vlad/anaconda/include/python2.7 -c agents3.c -o build/temp.macosx-10.5-x86_64-2.7/agents3.o
agents3.c:2273:52: error: use of undeclared identifier 'NPY_C_CONTIGUOUS'
    __pyx_t_2 = ((!(PyArray_CHKFLAGS(__pyx_v_self, NPY_C_CONTIGUOUS) != 0)) != 0);
                                                   ^
agents3.c:2311:52: error: use of undeclared identifier 'NPY_F_CONTIGUOUS'
    __pyx_t_1 = ((!(PyArray_CHKFLAGS(__pyx_v_self, NPY_F_CONTIGUOUS) != 0)) != 0);
                                                   ^
agents3.c:2474:42: error: no member named 'descr' in 'struct tagPyArrayObject'
  __pyx_t_4 = ((PyObject *)__pyx_v_self->descr);
                           ~~~~~~~~~~~~  ^
agents3.c:4026:27: error: no member named 'base' in 'struct tagPyArrayObject'
  Py_XDECREF(__pyx_v_arr->base);
             ~~~~~~~~~~~  ^
/Users/vlad/anaconda/include/python2.7/object.h:823:34: note: expanded from macro 'Py_XDECREF'
#define Py_XDECREF(op) do { if ((op) == NULL) ; else Py_DECREF(op); } while (0)
                                 ^
agents3.c:4026:27: error: no member named 'base' in 'struct tagPyArrayObject'
  Py_XDECREF(__pyx_v_arr->base);
             ~~~~~~~~~~~  ^
/Users/vlad/anaconda/include/python2.7/object.h:823:64: note: expanded from macro 'Py_XDECREF'
#define Py_XDECREF(op) do { if ((op) == NULL) ; else Py_DECREF(op); } while (0)
                                                               ^
/Users/vlad/anaconda/include/python2.7/object.h:772:24: note: expanded from macro 'Py_DECREF'
        --((PyObject*)(op))->ob_refcnt != 0)            \
                       ^
agents3.c:4026:27: error: no member named 'base' in 'struct tagPyArrayObject'
  Py_XDECREF(__pyx_v_arr->base);
             ~~~~~~~~~~~  ^
/Users/vlad/anaconda/include/python2.7/object.h:823:64: note: expanded from macro 'Py_XDECREF'
#define Py_XDECREF(op) do { if ((op) == NULL) ; else Py_DECREF(op); } while (0)
                                                               ^
/Users/vlad/anaconda/include/python2.7/object.h:775:34: note: expanded from macro 'Py_DECREF'
        _Py_Dealloc((PyObject *)(op));                  \
                                 ^
/Users/vlad/anaconda/include/python2.7/object.h:762:15: note: expanded from macro '_Py_Dealloc'
    (*Py_TYPE(op)->tp_dealloc)((PyObject *)(op)))
              ^
/Users/vlad/anaconda/include/python2.7/object.h:115:47: note: expanded from macro 'Py_TYPE'
#define Py_TYPE(ob)             (((PyObject*)(ob))->ob_type)
                                              ^
agents3.c:4026:27: error: no member named 'base' in 'struct tagPyArrayObject'
  Py_XDECREF(__pyx_v_arr->base);
             ~~~~~~~~~~~  ^
/Users/vlad/anaconda/include/python2.7/object.h:823:64: note: expanded from macro 'Py_XDECREF'
#define Py_XDECREF(op) do { if ((op) == NULL) ; else Py_DECREF(op); } while (0)
                                                               ^
/Users/vlad/anaconda/include/python2.7/object.h:775:34: note: expanded from macro 'Py_DECREF'
        _Py_Dealloc((PyObject *)(op));                  \
                                 ^
/Users/vlad/anaconda/include/python2.7/object.h:762:45: note: expanded from macro '_Py_Dealloc'
    (*Py_TYPE(op)->tp_dealloc)((PyObject *)(op)))
                                            ^
agents3.c:4035:16: error: no member named 'base' in 'struct tagPyArrayObject'
  __pyx_v_arr->base = __pyx_v_baseptr;
  ~~~~~~~~~~~  ^
agents3.c:4070:30: error: no member named 'base' in 'struct tagPyArrayObject'
  __pyx_t_1 = ((__pyx_v_arr->base == NULL) != 0);
                ~~~~~~~~~~~  ^
agents3.c:4093:44: error: no member named 'base' in 'struct tagPyArrayObject'
    __Pyx_INCREF(((PyObject *)__pyx_v_arr->base));
                              ~~~~~~~~~~~  ^
agents3.c:1065:37: note: expanded from macro '__Pyx_INCREF'
  #define __Pyx_INCREF(r) Py_INCREF(r)
                                    ^
/Users/vlad/anaconda/include/python2.7/object.h:767:18: note: expanded from macro 'Py_INCREF'
    ((PyObject*)(op))->ob_refcnt++)
                 ^
agents3.c:4094:41: error: no member named 'base' in 'struct tagPyArrayObject'
    __pyx_r = ((PyObject *)__pyx_v_arr->base);
                           ~~~~~~~~~~~  ^
11 errors generated.
error: command 'gcc' failed with exit status 1
Vlads-MacBook-Pro:program vlad$ 

What should I do? Is it OK to leave the deprecated API call as it is? It tries to acces the base field -- but I am not doing it, it's Cython's fault. I simply converted a memoryview to a numpy array. Is there another, cleaner/safer way of doing it?

Hamrah

FWIW, I get the same warnings. It hasn't broken anything yet, so I'm ignoring them for now. – Callisto

move define_macros and include_dirs to setup – Lurid

Answer 1

Just for a further reference, cython online docs says this is because Cython is using a deprecated Numpy API, and for the time being, it's just a warning that we can ignore.

Polley

Thank you! I chose your answer because it's not only your guess, but you brought evidence. :) – Hamrah

OK, but the warning is written to stdout, so it's annoying to filter. Can one configure cython to not report warnings? – Subject

@Subject If you're using cython magic in ipython then you can use %%cython --compile-args=-w to disable warnings. – Purge

The online doc section has moved to docs.cython.org/en/latest/src/userguide/… – Scurrilous

To summarize above doc: as of 2021, version cython 3.0 (still pre-release at the time of writing) addresses that issue, with earlier cython version keep ignoring the warning. – Shurlock

Answer 2

Cython documentation says:

In Cython 3.0, you can get rid of this warning by defining the C macro NPY_NO_DEPRECATED_API as NPY_1_7_API_VERSION in your build:

# distutils: define_macros=NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION

...

With older Cython releases, setting this macro will fail the C compilation, because Cython generates code that uses this deprecated C-API. However, the warning has no negative effects even in recent NumPy versions including 1.18.x. You can ignore it until you (or your library’s users) switch to a newer NumPy version that removes this long deprecated API, in which case you also need to use Cython 3.0 or later. Thus, the earlier you switch to Cython 3.0, the better for your users.

As of February 2023, Cython 3.0 has not been released, so that this advice to upgrade to Cython 3.0 to remove the warning didn't age well.

You can ignore it advice isn't the best practice either because it causes developer warning fatigue, apathy and ignorance, which leads to bugs.

In earlier Cython versions this warning can be disabled by defining C preprocessor macro NPY_NO_DEPRECATED_API=1, e.g.:

# distutils: define_macros=NPY_NO_DEPRECATED_API=1

I use this method with cython-0.29.32 and numpy-1.23.4, no issues in the subset of functionality I use.

Miser

Excellent answer! Has both the 'why' and the 'how' parts. – Unbelief

Cython 3.0 has been released this July. – Friedlander

Answer 3

I also get the same warnings, and I'd say it's normal.

With the numpy C API you need to put a line in front of the C script if you don't like this warning, but all it does is tell the compiler to ignore the "deprecated" message - it seems to work the same either way.

I'm guessing the Cython compiler doesn't put this line of code when it generates the C code, and I don't think that's important.

Osteitis

Answer 4

Assuming one wishes to hide the deprecation warning, the following compiler flag can be implemented with clang: extra_compile_args=['-Wno-#warnings'].

For gcc, extra_compile_args=['-Wno-cpp'] achieves the same.

Of course this also hides other preprocessor directive warnings.

Waft

That's bad habit to deprecate warnings – Slambang