How to type a function in function argument in cython

Question

I have made a function for doing optimization in python (let's call it optimizer). It requires the function to be optimized (let's call it objective) as one of the function arguments. objective is a function that accepts a one-dimensional np.ndarray and return a float number (which is the same as double in C++?).

I have read this post, but I'm not sure if it is actually the same problem as mine and when I use ctypedef int (*f_type)(int, str), but I get the error Cannot convert 'f_type' to Python object during compilation. Does it only work for C functions? How do I type a python function?

Edit: How my code looks like:

cpdef optimizer(objective, int num_particle, int dim,
             np.ndarray[double, ndim=1] lower_bound,
             np.ndarray[double, ndim=1] upper_bound):

    cdef double min_value
    cdef np.ndarray[double, ndim=2] positions = np.empty((num_particle,dim), dtype=np.double)
    cdef np.ndarray[double, ndim=1] fitness = np.empty(num_particle, dtype=np.double)
    cdef int i, j

    # do lots of stuff not shown here
    # involve the following code:
    for i in range(num_particle):
        fitness[i] = objective(positions[i])

    return min_value

I want to know if it is possible to type objective to make the code run faster.

Beaded

need more code to understand your problem. – Rhona

Answer 1

I get the error message

Cannot convert Python object argument to type 'f_type'

which I think makes a lot more sense than the one you claim to get - you're trying to pass a Python object into the function. Please make sure that the error messages you report are the ones your code actually generates. Your description of the types that objective takes also does not match the code you show.

---

However, in general: no you cannot give your objective function a type specifier to speed it up. A generic Python callable carries a lot more information than a C function pointer (e.g. reference counts, details of any closure captured variables, etc).

A possible alternative approach would be to inherit from a cdef class with an appropriate cdef function, so you can at least get the appropriate performance in specific cases:

# an abstract function pointer class
cdef class FPtr:
    cdef double function(self,double[:] x) except? 0.0:
        # I'm assuming you might want to pass exceptions back to Python - use 0.0 to indicate that there might have been an error
        raise NotImplementedError()

# an example class that inherits from the abstract pointer type    
cdef class SumSq(FPtr):
    cdef double function(self,double[:] x) except? 0.0:
        cdef double sum=0.0
        for i in range(x.shape[0]):
            sum += x[i]**2
        return sum

# an example class that just wraps a Python callable
# this will be no faster, but makes the code generically usable
cdef class PyFPtr(FPtr):
    cdef object f
    def __init__(self,f):
        self.f = f

    cdef double function(self,double[:] x) except? 0.0:
        return self.f(x) # will raise an exception if the types don't match

def example_function(FPtr my_callable):
    import numpy as np
    return my_callable.function(np.ones((10,)))

Using this example_function(SumSq()) works as expected (and with Cython speed); example_function(PyFPtr(lambda x: x[0])) works as expected (without Cython speed in the callable); example_function(PyFPtr(lambda x: "hello")) gives a type error as expected.

Theorem