Tool (or combination of tools) for reproducible environments in Python

Question

I used to be a java developer and we used tools like ant or maven to manage our development/testing/UAT environments in a standardized way. This allowed us to handle library dependencies, setting OS variables, compiling, deploying, running unit tests, and all the required tasks. Also, the scripts generated guaranteed that all the environments were almost equally configured, and all the task were performed in the same way by all the members of the team.

I'm starting to work in Python now and I'd like your advice in which tools should I use to accomplish the same as described for java.

Worn

Answer 1

1. virtualenv to create a contained virtual environment (prevent different versions of Python or Python packages from stomping on each other). There is increasing buzz from people moving to this tool. The author is the same as the older working-env.py mentioned by Aaron.

2. pip to install packages inside a virtualenv. The traditional is easy_install as answered by S. Lott, but pip works better with virtualenv. easy_install still has features not found in pip though.

3. scons as a build tool, although you won't need this if you stay purely Python.

4. Fabric paste, or paver for deployment.

5. buildbot for continuous integration.

6. Bazaar, mercurial, or git for version control.

7. Nose as an extension for unit testing.

8. PyFit for FIT testing.

Weariful

Answer 2

I also work with both java and python. For python development the maven equivalent is setuptools (http://peak.telecommunity.com/DevCenter/setuptools). For web application development I use this in combination with paster (http://pythonpaste.org/) for the deployment process

Tourer

setuptools is a pain for me. – Electrokinetic

Answer 3

Other than easy_install?

For our Linux servers, we use easy_install and yum.

For our Windows development laptops, we use easy_install and a few MSI's for some projects.

Most of the Python libraries we use are source-only, so we can use the same distribution on all boxes. If we could have a network shared device, we'd put them all there. Sadly, our infrastructure is kind of scattered, so we have to either move .TAR files around or redo the installs to rebuild the environments.

In a few cases (e.g., PIL), we have to recompile and check the version numbers.

Apiece

Answer 4

You will want easy_setup to get the eggs (roughly what Maven calls an artifact).

For setting up your environment, have a look at working-env.py

Python is not compiled but you can put all files for a project in an egg. This is done with setuptools

For CI, check this answer.

Prescott

Answer 5

We would be remiss not to also mention Paver, which was created by Kevin Dangoor of TurboGears fame. The project is still in alpha, but it appears very promising. A snippet from the project page:

Paver is a Python-based build/distribution/deployment scripting tool along the lines of Make or Rake. What makes Paver unique is its integration with commonly used Python libraries. Common tasks that were easy before remain easy. More importantly, dealing with your applications specific needs and requirements is now much easier.

Institution

Answer 6

I do exactly this with a combination of setuptools and Hudson. I know Hudson is a java app, but it can run Python stuff just fine.

Andes

Answer 7

You might want to check our Devenv. It allows you to standardize the build environments for development, QA and UAT. It's free as in "free beer".

HTH

Revell