How can use CFFI to call an existing C function given the source code?

Question

I have a C source/header file that are part of a bigger project. I would like to test this as a unit, independent of the real project. While it would be possible to do this in C by creating a new project with a different main(), I would like to see if I can use Python (3) and its frameworks (eg. nose) to accelerate the construction of tests, use existing reporting frameworks, etc.

I was under the impression that I could do this with CFFI. Here's a sample C file:

// magic.c
// Implementation of magic.
int add(int a, int b)
{
    return a;
}

The header:

// magic.h
// Add two numbers (where a + b is not greater than INT_MAX).
int add(int a, int b);

Here's a script that just tries to compile it so I can call some functions:

# cffi_test.py
import cffi

INCLUDE_DIRS = ('.',)

SOURCES = ('magic.c',)

ffi = cffi.FFI()

ffi.set_source(
    '_magic_tests',
    '#include "magic.h"',
    include_dirs = INCLUDE_DIRS,
    sources = SOURCES,
    libraries = [],
    )

ffi.compile()

Ultimately I plan to have this be part of the setup before a set of unit tests eg. a pure Python function test_add() will call and check the result of the C function add() via the ffi object, which is constructed in the test setup.

The above script seems to work; it runs without error, it creates a _magic_tests.c file, a _magic_tests.cp35-win32.pyd file, and a Release directory. I can also import _magic_tests without an error.

But I can't figure out how to actually call a C function via CFFI. I can't find any documentation for the set_source() function, and it seems pretty integral to the whole process. The overview mentions it a lot, but the reference contains zero occurrences of it. The docs do have a section on calling functions, but it refers to some lib object without showing how it's created. If I look at the previous example there's a lib object created from ffi.dlopen(), but I don't see how to apply that to something that CFFI itself is producing.

My big question (ie. my X problem) is:

• Is CFFI a reasonable tool to use for calling and testing C functions in a cross platform (Windows 7-10, Linux, OS X) way, and if it is, how?

The questions arising from my current approach (ie. my Y problems) are:

• Where is the documentation for set_source()? How can I find out what arguments it takes?
• How do I produce lib objects that contain the functions I want to call?
• Is this the easiest way to use CFFI to call a C function? I do not particularly need or want a shared library or redistributable package to be produced; if it has to happen, that's fine, but it's not necessary. What other approaches could I try?

My current setup is:

• OS: Windows 10
• Python: CPython 3.5.1 32 bit
• Pip: 8.1.2
• CFFI: 1.6.0
• C compiler: whatever comes with Visual C++ Build Tools 2015, linked from this MSDN post

I am using CFFI and pycparser from Christoph Gohlke's repository.

Clubbable

Answer 1

For a project of mine, I use cffi to test my C code. IMHO cffi is a great tool to generate python bindings for C code and therefore think that it is a reasonable tool to use for calling and testing C functions from python. However, your code will only be as cross platform as the C code is, since you have to compile the binding for every platform.

Below you can find a few references to the documentation that should answer your questions. Additionally I wrote some example code to illustrate how you would use cffi. For a larger example, you can find my project at https://github.com/ntruessel/qcgc/tree/master/test.

• The documentation for set_source() can be found here https://cffi.readthedocs.io/en/latest/cdef.html

• https://cffi.readthedocs.io/en/latest/overview.html explains how you can use CFFI, I recommend API, out-of-line.

Four your example, build_magic_tests.py would look something like this:

from cffi import FFI

ffibuilder = FFI()

# For every function that you want to have a python binding,
# specify its declaration here
ffibuilder.cdef("""
    int add(int a, int b);
                """)

# Here go the sources, most likely only includes and additional functions if necessary
ffibuilder.set_source("magic_tests",
    """
    #include "magic.h"
    """, sources=["magic.c"])

if __name__ == "__main__":
    ffibuilder.compile()

To generate the magic_tests module, you have to run python build_magic_tests.py. The generated module can be imported and used like this:

from magic_tests import ffi, lib

def run_add():
    assert 4 == lib.add(4, 5)

Lully

Great answer! My only further question is: it seems like there's some duplication between what's in the header file and what's in the ffibuilder.cdef() call; that is, I'd be declaring the function twice, and there's a risk it could get out of sync or introduce an error. Do you think there's a way to reduce that duplication? – Clubbable

Unfortunately, I think this is currently impossible, at least I did not find a way to do this. IMHO this is one of the major disadvantages of cffi. One could try to auto-generate the whole build_magic_tests.py using yet another script. However, this script has to work around the limitations of the ffibuilder.cdef() method. – Lully

Well, that's annoying but not a deal-breaker. It's still worth it to have access to Python's unit testing ecosystem. (And it's still less boilerplate than any C unit testing framework.) – Clubbable

I thought of one more question: how might you do cleanup? At the moment you import the module after compiling it, and you'd have to unimport it if you wanted to ensure a clean slate for every test. ffi.compile() returns the name of the .pyd file, which could be deleted, but there's a whole bunch of other stuff it generates for compilation which is compiler dependent. Do you know if there's a way to get at the magic_tests.lib object directly from CFFI calls? – Clubbable

Ah, it looks like I'm talking about what CFFI calls "API, in-line" ie. the verify() function. It's deprecated, so I'll just have to hack around it. – Clubbable

I felt like the quality of this answer deserved a little more rep than what you'll get for such a niche issue, and it really helped me out, so I added a bounty. – Clubbable