Convert Python program to C/C++ code? [closed]

Question

is it possible to convert a Python program to C/C++?

I need to implement a couple of algorithms, and I'm not sure if the performance gap is big enough to justify all the pain I'd go through when doing it in C/C++ (which I'm not good at). I thought about writing one simple algorithm and benchmark it against such a converted solution. If that alone is significantly faster than the Python version, then I'll have no other choice than doing it in C/C++.

Balfour

As much as Python loses on benchmarks, keep in mind that that 50x or 100x slowdown is still negible if the calculation finishes in a few seconds in Python, and not even true when you do a lot of I/O or have a horrible algorithm. Rather than asking "how much slower is Python?" you should ask "is Python fast enough?" (and it most propably is, honestly) - that's also faster than benchmarking or asking here. – Myrtismyrtle

Implementing an algorithm in python is quite fast and straight forward...you simply have to do it and then check if it is fast enough. Most times you can optimize the algorithm to run much faster using different data structures(dict/sets instead of lists...) or different operations. Anyway optimization should occur after you have already implemented a first draft of the algorithm and benchmarked/profiled it. – Clot

@delnan: in my case it's all about computation time. If the C variant needs x hours less, then I'd invest that time in letting the algorithms run longer/again. I simply want to find out (roughly) how much slower Python would be - if it's just a couple of hours I certainly wouldn't use a language I'm not comfortable with (you can ruin the best solutions to problems with bad implementations :P). – Balfour

@delnan's right about Python probably being fast enough for many things. Even when it slower, the ease of devleopment, maintenance, and future enhancement are important factors to consider. – Romanesque

"x hours"? How big is this? Have you benchmarked an implementation? Do you have measurements? Have you profiled the implementation? Or are you trying to prematurely optimize the solution? – Jodijodie

@Myrtismyrtle if you don't care about cloud costs, sureee – Rooks

Python may be fast enough, is it readable enough? Have you ever seen numpy code of the complex algorithm? – Lute

Found a useful tool to convert python to c++. It helped me in converting complex python code - javainuse.com/py2cpp – Enloe

Answer 1

Yes. Look at Cython. It does just that: Converts Python to C for speedups.

Shirlyshiroma

Of course that won't save you anything unless you add a bunch of cdef declarations and thereby introduce static typing (otherwise you just juggle opaque PyObject * stuff). And it will never get quite as fast as plain C because it's usually interfacing with Python (100% or more? only for plain numerical code that doesn't interface with Python at all for the most time!). But other than that, yes, it can get you a pretty devent speedup. – Myrtismyrtle

@delnan: In fact, it does save you something. Most pure Python code will be faster after compiled. But yes, with the cdefs and static typing you really start seeing differences. And the interfacing with Python you get in all cases where you use C from Python. – Shirlyshiroma

Answer 2

If the C variant needs x hours less, then I'd invest that time in letting the algorithms run longer/again

"invest" isn't the right word here.

1. Build a working implementation in Python. You'll finish this long before you'd finish a C version.

2. Measure performance with the Python profiler. Fix any problems you find. Change data structures and algorithms as necessary to really do this properly. You'll finish this long before you finish the first version in C.

3. If it's still too slow, manually translate the well-designed and carefully constructed Python into C.

   Because of the way hindsight works, doing the second version from existing Python (with existing unit tests, and with existing profiling data) will still be faster than trying to do the C code from scratch.

This quote is important.

Thompson's Rule for First-Time Telescope Makers
It is faster to make a four-inch mirror and then a six-inch mirror than to make a six-inch mirror.

Bill McKeenan
Wang Institute

Jodijodie

Regardless of the tremendous score, I do not see how does this answer the question. – Lute

@Audrius Meskauskas, it doesn't, it obviates it, making the question's very premise unnecessary. In other words, it explains how to do something better than what was asked, instead of what was asked. – Careycarfare

As for me, I would have upvoted it for the quote at the end, alone. That's tremendously insightful, when extrapolated to programming in C. I'm in the middle of a fast_malloc() implementation in C. I'm 4 weeks in on a project I thought would take a couple days. The majority of my time has been spent on support structure for the code, not the code itself. If building a shed is the goal, most of my effort is spent on building the nails, the hammer, the saw, and the chemicals to make the paint, so I can then make the can in which to mix the paint, and the paint stick too, and then the paint. – Careycarfare

If you pay for something, and its' total is too much, then you count every dollar to reduce it. C++ uses less memory and CPU resources. Maybe the GPU is fine to weave in across languages, due to GPU latency loading the array for certain types of problems (or AI for heavy solution to any problem) – Portwin

Answer 3

Shed Skin is "a (restricted) Python-to-C++ compiler".

From the docs:

Shed Skin is an experimental compiler, that can translate pure, but implicitly statically typed Python (2.4-2.6) programs into optimized C++. It can generate stand-alone programs or extension modules that can be imported and used in larger Python programs.

Besides the typing restriction, programs cannot freely use the Python standard library (although about 25 common modules, such as random and re, are currently supported). Also, not all Python features, such as nested functions and variable numbers of arguments, are supported.

For a set of a 75 non-trivial programs (at over 25,000 lines in total (sloccount)), measurements show a typical speedup of 2-200 times over CPython.

Triclinic

+1 one advantage of Shed Skin is type inference: if it is possible to go guess variable types from the program flow, dynamic type-checking is avoided. This typically leads to shorter C++ code that it is actually possible to read and compiles to faster programs. – Sarre

There is also Python → 11l → C++ transpiler, which is also a restricted Python to C++ compiler, but it supports some Python features, which is not supported with Shed Skin (e.g. nested functions/closures). – Houlberg

Answer 4

Just came across this new tool in hacker news.

From their page - "Nuitka is a good replacement for the Python interpreter and compiles every construct that CPython 2.6, 2.7, 3.2 and 3.3 offer. It translates the Python into a C++ program that then uses "libpython" to execute in the same way as CPython does, in a very compatible way."

Barometrograph

This project is so much more mature than other similar options. It's funny it creates the binary with an .exe extension on OSX even though it is a perfectly normal OSX Mach-O executable. Looks like it might be a good replacement for pyinstaller, py2exe, py2app, etc. The --recurse-*** flags are important to set properly though. – Helman

Nuitka is great, but the C/C++ code created is using PyObject which bind to the CPython-C-code-implementation. It does not produce idiomatic C-code. – Riegel

Answer 5

I know this is an older thread but I wanted to give what I think to be helpful information.

I personally use PyPy which is really easy to install using pip. I interchangeably use Python/PyPy interpreter, you don't need to change your code at all and I've found it to be roughly 40x faster than the standard python interpreter (Either Python 2x or 3x). I use pyCharm Community Edition to manage my code and I love it.

I like writing code in python as I think it lets you focus more on the task than the language, which is a huge plus for me. And if you need it to be even faster, you can always compile to a binary for Windows, Linux, or Mac (not straight forward but possible with other tools). From my experience, I get about 3.5x speedup over PyPy when compiling, meaning 140x faster than python. PyPy is available for Python 3x and 2x code and again if you use an IDE like PyCharm you can interchange between say PyPy, Cython, and Python very easily (takes a little of initial learning and setup though).

Some people may argue with me on this one, but I find PyPy to be faster than Cython. But they're both great choices though.

Edit: I'd like to make another quick note about compiling: when you compile, the resulting binary is much bigger than your python script as it builds all dependencies into it, etc. But then you get a few distinct benefits: speed!, now the app will work on any machine (depending on which OS you compiled for, if not all. lol) without Python or libraries, it also obfuscates your code and is technically 'production' ready (to a degree). Some compilers also generate C code, which I haven't really looked at or seen if it's useful or just gibberish. Good luck.

Hope that helps.

Algo

What software do you use to compile from PyPy interpretation? – Hydrocellulose

Not specifically PyPy, just .py scripts. Nuitka if you want "C/C++ executable's, or C/C++ source code" and PyInstaller if you just want an executable (easier). There's also py2exe but I had less success with it, though I'm sure things have improved. PyInstaller is also cross platform, not just for Windows executable's (works with Linux, and Mac). Nuitka is unique because I think it's the only "compiler" that gives you usable source code back that you could in theory optimize further. There are a few others like bbFreeze, cx_Freeze, and py2app but I haven't tried them. Best of luck! – Algo

I have also found PyPy to run faster than Cython. In one test I actually found PyPy to be the same speed as a C++ version of the program (insertion sort). – Occupy

Answer 6

Another option - to convert to C++ besides Shed Skin - is Pythran.

To quote High Performance Python by Micha Gorelick and Ian Ozsvald:

Pythran is a Python-to-C++ compiler for a subset of Python that includes partial numpy support. It acts a little like Numba and Cython—you annotate a function’s arguments, and then it takes over with further type annotation and code specialization. It takes advantage of vectorization possibilities and of OpenMP-based parallelization possibilities. It runs using Python 2.7 only.

One very interesting feature of Pythran is that it will attempt to automatically spot parallelization opportunities (e.g., if you’re using a map), and turn this into parallel code without requiring extra effort from you. You can also specify parallel sections using pragma omp > directives; in this respect, it feels very similar to Cython’s OpenMP support.

Behind the scenes, Pythran will take both normal Python and numpy code and attempt to aggressively compile them into very fast C++—even faster than the results of Cython.

You should note that this project is young, and you may encounter bugs; you should also note that the development team are very friendly and tend to fix bugs in a matter of hours.

Claritaclarity

Note that "until 0.9.5 (included), Pythran was supporting Python 3 and Python 2.7. It now only supports Python 3." – Riegel

Answer 7

http://code.google.com/p/py2c/ looks like a possibility - they also mention on their site: Cython, Shedskin and RPython and confirm that they are converting Python code to pure C/C++ which is much faster than C/C++ riddled with Python API calls. Note: I haven’t tried it but I am going to..

Wilterdink

It seems that Py2C is still an unfinished project. It hasn't been updated in a few years, so it might be defunct. – Varletry

@Wilterdink Still pretty useful, thank you - it can be a good alt. Solution, thanks for posting this. – Nonbeliever

Answer 8

I realize that an answer on a quite new solution is missing. If Numpy is used in the code, I would advice to try Pythran:

http://pythran.readthedocs.io/

For the functions I tried, Pythran gives extremely good results. The resulting functions are as fast as well written Fortran code (or only slightly slower) and a little bit faster than the (quite optimized) Cython solution.

The advantage compared to Cython is that you just have to use Pythran on the Python function optimized for Numpy, meaning that you do not have to expand the loops and add types for all variables in the loop. Pythran takes its time to analyse the code so it understands the operations on numpy.ndarray.

It is also a huge advantage compared to Numba or other projects based on just-in-time compilation for which (to my knowledge), you have to expand the loops to be really efficient. And then the code with the loops becomes very very inefficient using only CPython and Numpy...

A drawback of Pythran: no classes! But since only the functions that really need to be optimized have to be compiled, it is not very annoying.

Another point: Pythran supports well (and very easily) OpenMP parallelism. But I don't think mpi4py is supported...

Gynarchy

Pythran uses the simplest of directives for OpenMP like #omp parallel for or, if you wrote the inner loop in a 1 liner, and want it to be parallel too, then you can do a #omp parallel for collapse(2) and done. There's absolutely no reason to use mpi4py since Pythran releases the GIL (like Cython nogil statements) and runs your CPU at 100% with OpenMP, if you don't tell it to do otherwise. It's a great library if you can handle compiling functions in the simplest sense... I dumped Numba because of the slow JIT compile time, and AOT compiled functions don't support parallel operations. – Amund