Cython for a Django app: would it work?

Question

Would compiling with cython work with a python 3.4 Django app, or could it be made to work without a huge amount of effort?

This answer - https://mcmap.net/q/49562/-how-do-i-protect-python-code-from-being-read-by-users - to a question about protecting python code prompted me to ask this question.

A similar question has been asked previously but with regards to improving performance: Using Cython with Django. Does it make sense?

Edmanda

You understand that Cython is not for packages, right? – Stokes

I did not. Is there some workaround to that? – Edmanda

@IgnacioVazquez-Abrams: What do you mean "Cython is not for packages"? – Cephalization

Superficially the answer is "yes". Just change "models.py" to "models.pyx", "views.py" to "views.pyx", etc and compile appropriately. Cython can cope with the vast, vast majority of Python code (and a quick test with the first part of the Django tutorial suggests it works fine). You won't gain much though. What are you actually hoping to accomplish? – Selfjustifying

Make the app more difficult to reverse engineer for clients who want to run the app server in their own data center, using the idea of https://mcmap.net/q/49562/-how-do-i-protect-python-code-from-being-read-by-users – Edmanda

@Selfjustifying do you believe that doing this will not contribute much towards that aim? If so, why? – Edmanda

@Edmanda "You won't gain much" was mostly about performance. Guessing at difficultly to reverse engineer: the code generated will have a lot of Python API calls, and have a lot of strings (as attribute names, and because this kind of web app deals with a lot of string) which might make it comparatively easy to reverse engineer for someone who's really committed. However, it will be harder than from Python code. It might not be a lot harder than reverse engineering ".pyc" bytecode files. This is all a bit of a guess though. – Selfjustifying

I can tell you with good certainty that it will at least make the app much more difficult to maintain. I'd say do not compile the whole code, only parts of it - only calculations, that you could isolate as standard (no Django) Python functions. – Glaudia

Instagram did this, both for some of their performance critical internal code and for the Django url dispatcher. It is possible. – Elbring

Answer 1

Yes, we have done it. But it point of consistent pain.

We make a commercial product which is installed on the customer premise to manage their Genesys power contact center. The core of the application is written in Django and we wanted to protect (limit) the code from inspection.

There is a speed improvement from running in native python but it is not a considerable difference. The improvement depends on the type of task, sometimes up to 30% sometimes minimal.

We run into issues from time to time where something works in Python but then it does not in Cython. I would not recommend this path unless you have a really good motivation.

Currently version runs on Python 3.5 with Django 1.11

Pandemonium

can you give exact example commandline and methodology to compile a dummy hello world django project using cython, and then run it? – Letdown

Just as an update we have found not everything works under Cython and we needed to add in some exclude rules to make the solution work. What we have found is that Django migrations and any Celery scheduled tasks just do not work. So these components are deployed as pure Python and it is not big deal. This has been in production now for over 4 years with enterprise customers (such as banks, and goverment agencies) and works well. – Pandemonium

A practical example would have been nice :0 – Jamima

Answer 2

I know It is too late to answer. Even though It might help. I have created a setup.py file in the project home directory.

from distutils.core import setup
from Cython.Build import cythonize
fileSet = set()
fileSet.add("app1/file1.py")
fileSet.add("app2/file2.py")
fileSet.add("app3/file3.py")
setup(
   ext_modules=cythonize(fileSet)
)

Scan your app directories and add files to the fileSet whatever you want to compile. file1.py, file2.py and file3.py are just examples only.

Finally, just run the setup.py file as below

python setup.py build_ext --inplace 

Then Cython stats compiling each file and makes it .so file. Example: app1/file1.so app2/file2.so app3/file3.so

These files are shared object files and you cannot interpret manually. Delete all .py and .pyc files. And then run your project as

python manage.py runserver

or you can host these binaries in your production server. I tried on NGINX, uWSGI.

Good Luck.

Periwinkle

I didn't notice any difference in performance as of now. At present we don't have more number of users. But the application is working fine. May be we can find it by generating some load. – Periwinkle

So you can convert full Django project this way? Py files of course – Deryl

Answer 3

Please checkout djcompiler package that compiles django projects using Cython. https://github.com/abdoohossamm/djcompiler

run pip install djcompiler

then head to project directory and run djcompiler buildfile Edit your configurations in .djcompiler file. run djcompiler compile

Claiborne

You should probably make it clear that this is your project - there's no problem with linking to it but you're expected to disclose when you're promoting your own stuff – Selfjustifying