How to introspect a function defined in a Cython C extension module

Question

Python's inspect module doesn't seem to be able to inspect the signatures of "built-in" functions, which include functions defined in C extension modules, like those defined by Cython. Is there any way to get the signature of a Python function you have defined in such a module, and specifically in Cython? I am looking to be able to find the available keyword arguments.

MWE:

# mwe.pyx
def example(a, b=None):                                                                                                                                                       
    pass       

and

import pyximport; pyximport.install()                                                                                                                                         
import mwe                                                                                                                                                                    
import inspect                                                                                                                                                                

inspect.signature(mwe.example)   

yields:

Traceback (most recent call last):                                                                                                                                           
  File "mwe_py.py", line 5, in <module>                                                                                                                                      
    inspect.signature(mwe.example)                                                                                                                                           
  File "/nix/store/134l79vxb91w8mhxxkj6kb5llf7dmwpm-python3-3.4.5/lib/python3.4/inspect.py", line 2063, in signature                                                         
    return _signature_internal(obj)                                                                                                                                          
  File "/nix/store/134l79vxb91w8mhxxkj6kb5llf7dmwpm-python3-3.4.5/lib/python3.4/inspect.py", line 1965, in _signature_internal                                               
    skip_bound_arg=skip_bound_arg)                                                                                                                                           
  File "/nix/store/134l79vxb91w8mhxxkj6kb5llf7dmwpm-python3-3.4.5/lib/python3.4/inspect.py", line 1890, in _signature_from_builtin                                           
    raise ValueError("no signature found for builtin {!r}".format(func))                                                                                                     
ValueError: no signature found for builtin <built-in function example>    

In Python 3.4.5 and Cython 0.24.1

Decretive

Why shouldn't it be possible? inspect.signature(all) (for example to extract the signature of all) works great <Signature (iterable, /)>. Please provide an minimal reproducible example so that answers can actually show you how to do it in your case. – Wahl

Possible duplicate of Python inspect.getargspec with built-in function – Anastomose

Also relevant: #1105323 – Anastomose

@Anastomose Thanks, I have seen these answers, but had wondered if Cython adds some magic to make this work. Sorry, probably should have included in the question – Decretive

@Wahl I have updated with an MWE. If you voted -1, would you mind reversing? – Decretive

Answer 1

I've retracted my duplicate suggestion (saying that it was impossible...) having investigated further. It seems to work fine with reasonably recent versions of Cython (v0.23.4) and Python 3.4.4.

import cython
import inspect
scope = cython.inline("""def f(a,*args,b=False): pass """)
print(inspect.getfullargspec(scope['f']))

gives the output

FullArgSpec(args=['a'], varargs='args', varkw=None, defaults=None, kwonlyargs=['b'], kwonlydefaults={'b': False}, annotations={})

---

Also mentioned in the documentation is the compilation option "binding" which apparently makes this detail more accessible (although I didn't need it).

---

I have a feeling that this may depend on improvements to inspect made relatively recently (possibly this fix) so if you're using Python 2 you're probably out of luck.

---

Edit: your example works if you using the binding compilation option:

import cython
@cython.binding(True)
def example(a, b=None):                                                                                                                                                       
    pass

I suspect that inline adds it automatically (but the code to do inline is sufficiently convoluted that I can't find proof of that either way). You can also set it as a file-level option.

Anastomose

Thanks David, I have updated the question with an MWE, doesn't seem to work with an import as opposed to inline function. – Decretive

@Decretive See edit - it's just a case of making sure you use binding – Anastomose

Thanks a heap! Exactly what I was looking for. – Decretive

getfullargspec and signature don't seem to get the keyword arguments correctly for cdef class methods, though they do for pyx module functions. The keyword arguments appear as if they are positional arguments. These functions do get the names of arguments correctly for module functions. – Maggi

@IoannisFilippidis I was just about to tell you that it might be worth filling a bug, but it looks like you've already done that... – Anastomose

Does anyone know if this can be done in python 2.7? getfullargspec works only from python 3 on – Glacialist

Answer 2

The answer above using the binding decorator works for me when running code that has been cythonized. But, when I was running that same code within a Django 2.2 app, the application would fail on start with an error that cython has no attribute 'binding'. To avoid this I have added this "special cython header" at the top of my file containing the cythonized function as documented here to achieve the same results.

# cython: binding=True

def example(a, b=None):                                                                                                                                                       
    pass

Gazette