Calling C functions in Python

Question

I have a bunch of functions that I've written in C and I'd like some code I've written in Python to be able to access those functions.

I've read several questions on here that deal with a similar problem (here and here for example) but I'm confused about which approach I need to take.

One question recommends ctypes and another recommends cython. I've read a bit of the documentation for both, and I'm completely unclear about which one will work better for me.

Basically I've written some python code to do some two dimensional FFTs and I'd like the C code to be able to see that result and then process it through the various C functions I've written. I don't know if it will be easier for me to call the Python from C or vice versa.

Dulcedulcea

Do you have a lib ? .so ? – Circumscription

Shared lib ? g++ -shared -Wl,-soname,mylib.so -o mylib.so my.o ? – Circumscription

Answer 1

If I understand well, you have no preference for dialoging as c => python or like python => c. In that case I would recommend Cython. It is quite open to many kinds of manipulation, specially, in your case, calling a function that has been written in Python from C.

Here is how it works (public api) :

The following example assumes that you have a Python Class (self is an instance of it), and that this class has a method (name method) you want to call on this class and deal with the result (here, a double) from C. This function, written in a Cython extension would help you to do this call.

cdef public api double cy_call_func_double(object self, char* method, bint *error):
    if (hasattr(self, method)):
        error[0] = 0
        return getattr(self, method)();
    else:
        error[0] = 1

On the C side, you'll then be able to perform the call like so :

PyObject *py_obj = ....
...
if (py_obj) {
    int error;
    double result;
    result = cy_call_func_double(py_obj, (char*)"initSimulation", &error);
    cout << "Do something with the result : " << result << endl;
}

Where PyObject is a struct provided by Python/C API After having caught the py_obj (by casting a regular python object, in your cython extension like this : <PyObject *>my_python_object), you would finally be able to call the initSimulation method on it and do something with the result. (Here a double, but Cython can deal easily with vectors, sets, ...)

Well, I am aware that what I just wrote can be confusing if you never wrote anything using Cython, but it aims to be a short demonstration of the numerous things it can do for you in term of merging.

By another hand, this approach can take more time than recoding your Python code into C, depending on the complexity of your algorithms. In my opinion, investing time into learning Cython is pertinent only if you plan to have this kind of needs quite often...

Hope this was at least informative...

Antineutron

Thanks! It looks like with the same amount of effort I could just recode in C, so in this case I don't really have to use Cython. But I suspect I'll find this info very useful in the future. – Dulcedulcea

Is this C or C++? cout is part of <iostream>, isn't it? – Levirate

Answer 2

You should call C from Python by writing a ctypes wrapper. Cython is for making python-like code run faster, ctypes is for making C functions callable from python. What you need to do is the following:

1. Write the C functions you want to use. (You probably did this already)
2. Create a shared object (.so, for linux, os x, etc) or dynamically loaded library (.dll, for windows) for those functions. (Maybe you already did this, too)
3. Write the ctypes wrapper (It's easier than it sounds, I wrote a how-to for that)
4. Call a function from that wrapper in Python. (This is just as simple as calling any other python function)

Pastel

given that OP is the author of both C and Python code and OP prefers to write Python code, it is not clear what should be used here. Cython would allow to pull more code from C. – Polytrophic

+1 for pertinent comment. And Cython is not only a matter of performance. It is a complex and powerful tool. It allows you to mix C code with python code, so if you happen to code in C it can be the perfect bridge between C and Python. That's to say, you can mix statically (strong typed variables) with dynamically typed ones. And switch from C (PyObject) representation to Python (object) representation by a simple cast, in both ways. – Antineutron

Very nice tutorial at the link, but I did have to change int sum to int sum = 0 for it to work correctly. – Vladikavkaz

@Vladikavkaz Thanks, fixed. – Pastel

Thanks for the tutorial link - really nice. The only thing that I would change is the name of the Python function in the 'Basics' section, as you have overloaded 'our_function' which could be a bit confusing. – Purism

in the 3-rd part, when you're doing the following: array_type = ctypes.c_int * num_numbers result = _sum.our_function(ctypes.c_int(num_numbers), array_type(*numbers)), does this create a copy of the numbers list, or not? – Chaschase

@Chaschase That creates a new ctypes array containing the values from the list, and passes that array to the function. – Pastel

Answer 3

Well, here you are referring to two below things.

1. How to call c function within from python (Extending python)
2. How to call python function/script from C program (Embedding Python)

For #2 that is 'Embedding Python'

You may use below code segment:

#include "python.h"

int main(int argc, char *argv[]) {   
Py_SetProgramName(argv[0]);  /* optional but recommended */   
Py_Initialize();   
PyRun_SimpleString("from time import time,ctime\n"
    "print 'Today is',ctime(time())\n");
/*Or if you want to run python file within from the C code*/       
//pyRun_SimpleFile("Filename");    
Py_Finalize();   
return 0; }

For #1 that is 'Extending Python' Then best bet would be to use Ctypes (btw portable across all variant of python).

>> from ctypes import *

>> libc = cdll.msvcrt

>> print libc.time(None)

>> 1438069008

>> printf = libc.printf

>> printf("Hello, %s\n", "World!")

>> Hello, World!
14

>> printf("%d bottles of beer\n", 42)

>> 42 bottles of beer
19

For detailed guide you may want to refer to my blog article:

Topic

Answer 4

It'll be easier to call C from python. Your scenario sounds weird - normally people write most of the code in python except for the processor-intensive portion, which is written in C. Is the two-dimensional FFT the computationally-intensive part of your code?

Lagting

It's the least computationally intensive part... it just turns out that writing the code in Python is far more convenient that writing it in C. I'm hoping it's quicker to write a wrapper for the function than re-write the whole thing in C. – Dulcedulcea

Answer 5

There's a nice and brief tutorial on this from Digital Ocean here. Short version:

1. Write C Code
You've already done this, so super short example:

#include <stdio.h>

int addFive(int i) {
    return i + 5;
}

2. Create Shared Library File
Assuming the above C file is saved as c_functions.c, then to generate the .so file to call from python type in your terminal: cc -fPIC -shared -o c_functions.so c_functions.c

3. Use Your C Code in Python!
Within your python module:

# Access your C code
from ctypes import *
so_file = "./c_functions.so"
c_functions = CDLL(so_file)

# Use your C code
c_functions.addFive(10)

That last line will output 15. You're done!

Imeldaimelida

Answer 6

The answer from BLimitless quoting Digital Ocean is fine for simple cases, but it defaults to allowing int type arguments only. If need a different type for your input argument, for example to a float type, you need to add this:

c_functions.addFive.argtypes=[ctypes.c_float]

And if you change the output argument, for example to a float type you need this:

c_functions.addFive.restype=ctypes.c_float

J