Python distributions and environments for scientific computing

Question

I apologize upfront if this question is too broad. I come from the MATLAB world and have relatively little experience with Python.

After having spent some time reading about several Python-based environments and distributions for scientific computing, I feel that I still don't fully understand the landscape of solutions or the precise relationship between some notable packages, including:

• SciPy
• Spyderlib
• Pythonxy
• Enthought Python Distribution
• Sage

More specifically:

• Do any of the above packages provide similar functionality? Do they complement each other?
• Does the installation of any of them include or require the installation of any of the others? If so, which ones include or require which?

Less importantly, are there any other packages similar to the ones above that provide similar functionality?

Thanks in advance

Colville

Answer 1

Scientific computing with Python is taking a plain vanilla language and bolting on a bunch of modules, each of which implement some aspect of the functionality of MATLAB. As such the experience with Python scientific programming is a little incohesive c.f. MATLAB. However Python as a language is much cleaner. So it goes.

The basic necessary modules for scientific computing in Python are Numpy, Matplotlib, SciPy and if you are doing 3d plotting, then Mayavi/VTK. These modules all depend on Numpy.

Numpy Implements a new array type that behave similar to MATLAB arrays (i.e. fast vector calculations). It also defines a load of functions to do these calculations which are usually named the same as similar functions in MATLAB.

Matplotlib Allows for 2d plotting with very similar commands to MATLAB. Matplotlib also defines pylab, which is a module that - with a single import - brings most of the Numpy and Matplotlib functions into the global namespace. This is useful for rapid/interactive scripting where you don't want to be typing lots of namespace prefixes.

SciPy is a collection of Python modules arranged under the SciPy umbrella that are useful to scientists. Fitting routines are supplied in SciPy modules. Numpy is part of Scipy.

Spyder is a desktop IDE (based on QT) that loosely tries to emulate MATLAB IDE. It is part of the Python-XY distribution.

IPython provides an enhanced interactive Python shell which is useful for trying out code and running your scripts and interacting with the results. It can now be served to a web interface as well as the traditional console. It is also embedded in the Spyder IDE.

Distributions

Getting all these modules running on your computer can be time consuming and so there are a few distributions that package them (plus many other modules) up for you.

Python-XY, WinPython, Enthought and more recently Anaconda are all full package distributions that include all the core modules, although Enthought does not come with Spyder.

Sage is another programming environment which is served over the web or via a command line and also comes as a full package including lots of other modules. Traditionally it came as a VMWare image based on an install of Linux. Although you are writing Python in the Sage environment, it's a little different to ordinary Python programming, it kind of defines its own language and methodology based on Python.

If you are using Windows I would install WinPython. It installs everything that you need including Scipy and Spyder (which is the best replacement for MATLAB for Python IMHO) and because it is designed to be standalone it will not interfere with other installs of Python you may have on your system. If you are on OSX, Enthought is probably the best way to go - Spyder can be installed separately using e.g. MacPorts. For Linux you can install the components (Numpy, SciPy, Spyder, Matplotlib) separately.

I personally don't like the Sage way of working with Python 'hidden under the hood' but you may prefer that.

Steady

That was a great summary. Thanks a lot! – Colville

In addition to the above for the sake of completeness maybe some sort of machine learning package such as scikit-learn should be also included. BTW, latest version of Spyder works great on OSX. – Tyrolienne

I would add Pandas to the list of core scientific computing modules. It is included in the distributions you listed. Pandas uses the DataFrame object for storing and interacting with data. This should be familiar to those coming from R. – Root

Answer 2

Regarding the less important part of the question:

• Anaconda is another notable full-flechted distribution (similar to Enthought and Sage) coming with IPython, Spyder... It also let's you easily switch python versions/environments.
• PyCharm is another notable IDE, the one I tend to use.

Pentothal

Answer 3

This link may be usefull: https://www.cfa.harvard.edu/~ebresser/python/

It's the page of an astrophysicist at Harvard. It gives the point of view of someone switching from ITT-VIS IDL to python, on OS-X (but most tips also work on other operating systems).

EDIT: It seems the page was taken down. An alternative good introduction to python for a scientist/engineer is in this document (big PDF warning): http://stsdas.stsci.edu/perry/pydatatut.pdf Hope this one will not be taken down!

Litchi