How to distribute Python/Django App via RPM

Question

I've read over this question and I think that is a great start, but I'm looking for more insight.

I have written an application using Python 2.6 and Django 1.2.1. I would like to deploy this application on a system that does not have Python 2.6 or Django installed. The system itself doesn't have many things installed - sqlite3, pysqlite2, gcc, etc. The system does have Python 2.4.3, however.

What would be the most sane way to create RPMs of Python 2.6, Django, and my application itself (a standard Django app) so that each can be installed separately to allow for upgrades of different components?

I don't know much about creating source RPMs but I suspect that is information readily available. If there are any especially good resources for creating source RPMs, I would appreciate them too.

The system is ESX 4 Update 1, if it matters.

Thanks!

UPDATE: It also never occurred to me that distributing Python eggs or even using setup.py is a viable solution to content distribution. I would be interested in hearing about that too.

Kirstinkirstyn

Answer 1

Python packages are normally distributed with setuptools/distribute. Generally you can install every python package (and it's dependencies) with easy_install <package_name> (optionally with a specific version number). I would recommend you to use setuptools as our primary distribution method, because:

• it works on all platforms
• is the preferred way for Python packages in general
• .rpm or .deb install scripts can reuse parts of it too
• you can install this packages inside a virtualenv without the need of root privileges, conflict-free of the rest of the installed web applications on the server

A good resource for you to read is Deploying with Distribute. It focuses on Flask (another web framework), but simply replacing "Flask" with "Django" will do the trick :)

Another python tool which is great for distributing and deploying things is fabric, which you can use in addition to distribute (or without it, however you want). With fabric you can for example, bootstrap and install your application (inclusive dependencies) on N servers at once, manage several development, testing and production servers and update them all concurrently with a single command.

This article might help you to get started with Fabric: Deploying with Fabric

And additionally I would also recommend you to publish your repository (or mirror an existing one) on github or bitbucket for people who want to get the latest sources manually and install your application with python setup.py install afterwards. The setup.py install script is part of distribute which I've mentioned before.

Hollinger

How would it run my manage.py runserver? Where should I specify it? – Fortis

Answer 2

On top of all the normal issues encountered when packaging Python code, Django projects have a few other issues that need to be handled:

• Playing nice with system paths
• Playing nice with system facilities
• Handling translations
• Randomizing the secret key

I had to solve these issues for packaging Transifex, and I recommend you investigate the various bits that have put in place in order to make it work:

• Install under /usr/share/<projectname>
• Use execfile() to load settings from /etc/<projectname>
• The huge translation-handling runaround
• Replacement of the secret key in the post-install script
• All the other bloody changes and fudges that were required

Frederiksberg

I wouldn't install a web application under a system path, because than you might get troubles with different versions running at once or conflicting dependencies. Additionally, randomizing the secret key should be done during the configuration and not the installation (imaging several servers running in a cluster...). – Hollinger

If you have a cluster then you're going to use a tool such as puppet to deploy anyways, so they'll end up with the same key regardless. And being under a system path is fine since you need to go to great lengths to get more than one version of a package installed in the first place. – Frederiksberg

Answer 3

Refer to chapter 5 of the Python documentation. It will show how to deal with the bdist Python utility. With it, you'll be able to generate packages (or, how it says, "built distribution") of several types, includding RPM.

Scurlock

Answer 4

There are already RPM's of python 2.6 and django available - so those at least are (relatively) easy to grab and install via RPM. FOr creating an RPM for your project, there's a write up on some of the details using Distutils, that goes into the details of what you need in a spec file and what you invoke to make it happen.

There's also a note about creating RPM's from python on StackOverflow at Creating Python RPM that has a touch of detail as well.

From experience, packaging up python itself as an RPM is the trickiest of the critters, so taking advantage of what's already there is something I'd definitely recommend. For a few programs where I've seen this done, we have a Makefile (yeah, old school) run the setup.py bdist_rpm command on a continuous integration server, making the RPM available as the result of that build and immediately available on the CI server.

Opportune

The question is about creating an rpm of a django app you developed yourself, not about creating an rpm of django – Mameluke

Answer 5

I've written up a blog post on how to use setuptools and it's bdist_rpm command to create the rpm for you. So indeed, use setuptools, but indeed, distribute your app as an rpm. There's a few catches and weird under documented things in setuptools you can use to eventually end up in a nice workflow.

The key is you need a post install script that runs your migrations and collects your static files, that gets run after the rpm is installed

export DJANGO_SETTINGS_MODULE=accountpage.settings
mkdir -p /var/www/django/static && chown apache:apache /var/www/django/static
runuser -m apache -c 'echo "yes" | django-admin collectstatic'
django-admin migrate
service httpd restart

and you can tell setuptools to add this to the spec file before it creates the rpm by giving this option in the setup call:

options = {'bdist_rpm':{'post_install' : 'bdist_rpm_post_install.sh'}},

See the link above for a full working setup.py script, post install hook, and a testsuite that works with python setup.py test that will install all your dependencies locally, and run the django tests.

To have the rpm also automatically install it's dependencies, create a setup.cfg file which tells bdist_rpm about it's dependencies

[bdist_rpm]
requires = python >= 2.7
           django >= 1.8
           httpd
           Django-Select2 >= 4.3.1
           django-easy-select2
           django-model-utils
           pytz
           django-oauth-toolkit

Mameluke

Link to blog post is dead. – Stench