Continuous Integration System for a Python Codebase

Question

I am starting to work on a hobby project with a Python codebase and I would like to set up some form of continuous integration (i.e. running a battery of test-cases each time a check-in is made and sending nag e-mails to responsible persons when the tests fail) similar to CruiseControl or TeamCity.

I realize I could do this with hooks in most VCSes, but that requires that the tests run on the same machine as the version control server, which isn't as elegant as I would like. Does anyone have any suggestions for a small, user-friendly, open-source continuous integration system suitable for a Python codebase?

Nexus

Answer 1

We run Buildbot - Trac at work. I haven't used it too much since my codebase isn't part of the release cycle yet. But we run the tests on different environments (OSX/Linux/Win) and it sends emails — and it's written in Python.

Tuyere

Answer 2

One possibility is Hudson. It's written in Java, but there's integration with Python projects:

Hudson embraces Python

I've never tried it myself, however.

(Update, Sept. 2011: After a trademark dispute Hudson has been renamed to Jenkins.)

Deadfall

@Joe, good recommendation. @Nexus Don't let the fact that Hudson is written in Java scare you away if you're a python guy. It's dead simple to setup and run. It integrates with pylint and coverage.py. I have it running on my projects and love it. It's way simpler that BuildBot. To set it up and run Hudson it's one command. java -jar hudson.war Here is the best post I've seen for setting it up: rhonabwy.com/wp/2009/11/04/… – Willingham

Hudson being written in Java is as relevant as your the editor you use being written in C++! It's just a very well written, easy to setup and generally fully featured CI tool. Python can be very well integrated, nosetests --with-xunit --with-coverage, pylint with the "Violations" plugins, and "Cobertura" for coverage – Pone

@dbr: Not totally true. The language something is written in often influences the software it works with best... build systems especially. When I looked at Hudson ~2 years ago it wanted unit test output only in JUnit's XML format; my Python unit test suite didn't do that. But if my project were in Java, it would have worked out of the box. In any case, it looks like integration has come a long way in both Hudson and Python's tools since then, and I am thankful for that. – Deadfall

yep jenkins/hudson is perfectly fine, a breeze to set up. – Ideologist

Answer 3

Second the Buildbot - Trac integration. You can find more information about the integration on the Buildbot website. At my previous job, we wrote and used the plugin they mention (tracbb). What the plugin does is rewriting all of the Buildbot urls so you can use Buildbot from within Trac. (http://example.com/tracbb).

The really nice thing about Buildbot is that the configuration is written in Python. You can integrate your own Python code directly to the configuration. It's also very easy to write your own BuildSteps to execute specific tasks.

We used BuildSteps to get the source from SVN, pull the dependencies, publish test results to WebDAV, etcetera.

I wrote an X10 interface so we could send signals with build results. When the build failed, we switched on a red lava lamp. When the build succeeded, a green lava lamp switched on. Good times :-)

Brilliant

Answer 4

We use both Buildbot and Hudson for Jython development. Both are useful, but have different strengths and weaknesses.

Buildbot's configuration is pure Python and quite simple once you get the hang of it (look at the epydoc-generated API docs for the most current info). Buildbot makes it easier to define non-testing tasks and distribute the testers. However, it really has no concept of individual tests, just textual, HTML, and summary output, so if you want to have multi-level browsable test output and so forth you'll have to build it yourself, or just use Hudson.

Hudson has terrific support for drilling down from overall results into test suites and individual tests; it also is great for comparing test output between builds, but the distributed (master/slave) stuff is comparatively more complicated because you need a Java environment on the slaves too; also, Hudson is less tolerant of flaky network links between the master and slaves.

So, to get the benefits of both tools, we run a single instance of Hudson, which catches the common test failures, then we do multi-platform regression with Buildbot.

Here are our instances:

• Jython Hudson
• Jython buildbot

Iridium

sorry about that, I don't maintain that one :-) – Iridium

Answer 5

We are using Bitten wich is integrated with trac. And it's python based.

Labiche

Answer 6

TeamCity has some Python integration.

But TeamCity is:

• not open-source
• is not small, but rather feature rich
• is free for small-mid teams.

Rovner

Answer 7

I have very good experiences with Travis-CI for smaller code bases. The main advantages are:

• setup is done in less than half a screen of config file
• you can do your own installation or just use the free hosted version
• semi-automatic setup for github repositories
• no account needed on website; login via github

Some limitations:

• Python is not supported as a first class language (as of time of writing; but you can use pip and apt-get to install python dependencies; see this tutorial)

• code has to be hosted on github (at least when using the official version)

Soelch

Python is now fully supported in Travis CI: about.travis-ci.org/blog/… – Madlynmadman