Installing Python, virtualenv, NumPy, SciPy, matplotlib and IPython on Lion or Mountain Lion

Please note that the current version of this guide is available here. This page is no longer updated.

I decided to clear the cobwebs out of my  MacBook Pro (update 8/20/11: new MacBook Air in hand!) and start fresh with Lion, Python and all the other tools I regularly use (virtualenv, NumPy, SciPy, matplotlib and IPython). It was a lengthy process and I was unable to find a comprehensive walkthrough online, so I had to figure it out through trial and error (mostly error). I was surprised at the lack of available information; with such a major release I expected more complete documentation. So, once again quoting Tom Lehrer, I have a modest example here.

The important things we'll accomplish in this guide are (click any section to jump directly there):

1. A clean install of Mac OS 10.7 Lion
2. Installing Homebrew
3. Installing Python
4. Installing virtualenv
5. Installing NumPy, SciPy and matplotlib
6. Installing IPython and the qtconsole

As you may recall, this has been a somewhat more-than-trivial process for Lion's early adopters, as enough was changed behind the scenes to mess up some of the current versions of these tools. I'll detail how to get around those issues by getting cutting-edge distributions. Note that today's cutting-edge works on my machine; tomorrow's may introduce new bugs. When stable Lion-compatible versions of these programs are available, I strongly encourage you to use those instead.

If you would rather skip my commentary, you should be able to simply execute every line of code in sequence.

As always, remember that some of these lines will take you off the beaten path and could cause serious irrevocable damage to your computer! In particular, typing sudo gives you root access and therefore the ability to erase everything with a single keystroke. Please do not execute any code without understanding exactly what it does. In the worst case, I suppose you could always jump back to square one by wiping your hard drive and running a Lion clean install, but use caution nonetheless. YMMV.

Lion Clean Install

To begin, there has been some concern that users can't perform a clean install of Mac OS X 10.7 (Lion) without quite a bit of hacking around with physical media. I'm happy to report that isn't true -- as long as you performed a standard Lion install first. The OSX installer (App Store link) automatically creates a "Recovery HD" partition on your hard drive, just in case something goes wrong. The partition contains just two things: Disk Utility  and the ability to run the Lion installer. To access this recovery mode, hold Command-R while your computer is starting up. Use Disk Utility to erase your hard drive, then run the installer to re-download and load Lion. When you're done, you'll have a clean install of Lion. Download your favorite browser and you're good to go. I have to say, discovering this feature was a pleasant surprise -- hats off to Apple for thinking ahead on this one.

Next, install Xcode 4 -- its bundled compilers have to be available for the next step to work. Here's a link to it in the App Store; it's a free (but large) download. Note that like Lion, the download does not actually install the application. Instead, it puts an "Install Xcode" icon in your applications folder, which you must run to complete the installation. I don't understand this behavior of installing the installer, but it is what it is.

Homebrew

Next up, one of my new favorite tools -- Homebrew. The homepage says it pretty well: "Homebrew is the easiest and most flexible way to install the UNIX tools Apple didn't include with OS X." If you've messed around with MacPorts (or tried to do this sort of stuff  -- gasp -- by yourself), you'll be shocked by the drop-dead simplicity of this tool. Even installing it is a no-brainer; just fire up Terminal and execute the following line:

ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go)

That's it -- now you have Homebrew. Try running brew update and brew doctor at the command line to make sure it's working properly.

You'll need to add the Homebrew directory to your system path, in order to make sure that Homebrew-installed software is given higher priority than any other version. To do so, open your .bash_profile in your user directory (that's the ~ directory or, more verbosely, /Users/[your_user_name]) and add the following line (if you don't have a .bash_profile, create one with your favorite text editor -- you may need to do this in order to see it, as it will be a hidden file):

export PATH=/usr/local/bin:$PATH

If you haven't already, restart Terminal so it picks up the new path. Homebrew is generally very good about alerting you to any action you need to take after it finishes running, including path modifications. Make sure to pay attention to its output.

Python

Now on to the main event: Python. As of this writing, 2.7.3 is the most recent version. If that's no longer true as you read this, be sure to change any version-specific references! The following instruction will install Python, first installing any prerequisites. Mountain Lion users may have to install XQuartz; simply follow the instructions if prompted to do so. The --framework option tells it to build as a Framework, which has some downstream niceties, and --universal builds a universal (32/64 bit) version:

brew install python --framework --universal

This will install a version of disutils, but you must update your path in order to run it properly. Add the following to your .bash_profile (you may combine it with the previous .bash_profile edit, if you choose):

export PATH=/usr/local/share/python:$PATH

Once again, make sure you reload Terminal to update your path variable. Finally, you need to change Lion's symlink to point at your new Python install. Run the following three lines in sequence:

cd /System/Library/Frameworks/Python.framework/Versions
sudo rm Current
ln -s /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/Current

You will be prompted for your password after executing the sudo command.

To verify that your installation went as planned, type which python at the command line. You should see /usr/local/bin/python in response. Furthermore, if you type python, you should see Python 2.7.3 launch (type quit() to return to Terminal).

You should now have pip available on your system. If you discover that you don't have pip, simply execute:

easy_install pip

Note that depending on your permissions settings, you may have to prepend sudo to the front of many of these lines. Always remember that sudo gives you root access!! Therefore, in the interest of caution, I'm not going to include the sudo command here, but if your installs are failing because of permissions, you may have to add it.

virtualenv

Virtualenv is a tool that allows you to create isolated Python environments, each with its own set of packages and dependencies. This is useful for testing or managing package requirements (for example, if you build an application that is dependent on a certain version of a third-party package but another application requires a more recent version, you may break the first application by upgrading). This is not a required step, and all the commands below should work whether or not you are using virtualenv, so I'm listing it here for convenience only. The only difference will be that directories (such as that returned by which pip) will point to the virtualenv rather than /usr/local.

First, install virtualenv and virtualenvwrapper (a tool which makes working with virtualenv somewhat easier):

pip install virtualenv
pip install virtualenvwrapper

Next, source the virtualenvwrapper script:

source /usr/local/share/python/virtualenvwrapper.sh

This will create a hidden virtualenv directory at ~/.virtualenv. Now you can make your first virtual environment:

mkvirtualenv test1

Your new virtualenv test1 comes with a complete install of Python 2.7.3 and its own version of Pip. It is activated by default, so running any pip command will only impact this environment. Note that if you deactivate the virtualenv, you will lose access to any packages installed in it. You can switch between virtualenvs with the workon command.

In order to delete your test virtualenv, run rmvirtualenv test1. Also, to specify a different version of Python, just use mkvirtualenv -p python3.2 test1 (replacing python3.2 with the shell name of your preferred version).

NumPy

If pip installed properly (executing which pip should return /usr/local/share/python/pip), then you can install NumPy simply with:

pip install numpy

Check your work by executing:

python

followed by:

import numpy
print numpy.__version__
print numpy.__file__
quit()

The version should be 1.6.2 (as of July 2012), and the file should be somewhere in /usr/local/Cellar/....

SciPy

SciPy requires a Fortran compiler, which Lion lacks. To acquire one, use Homebrew:

brew install gfortran

Under Lion, install the stable version of SciPy (0.10) by running:

pip install scipy

Mountain Lion users will need to install the development version of SciPy (0.11) by executing the following line:

pip install -e git+https://github.com/scipy/scipy#egg=scipy-dev

As with NumPy, make sure you have everything working by running:

python

and then:

import scipy
print scipy.__version__
print scipy.__file__
quit()

matplotlib

On Lion, install matplotlib like any other package -- if you like, you can check your work once it finishes.

pip install matplotlib

Mountain Lion users will have to use the development version as of this writing:

pip install git+https://github.com/matplotlib/matplotlib.git#egg=matplotlib-dev

IPython

Installing IPython itself is a fairly straightforward pip command:

pip install ipython

Getting the new qtconsole to run takes a little more work -- but it's well worth it. The qtconsole is like running IPython on steroids -- my favorite feature is that matplotlib output can be displayed inline.

To begin, you'll need Nokia's qt library, which is available here. Make sure you download the library, not the SDK.

Once that's done, begin installing the prerequisites:

brew install pyqt

Note that after installing pyqt, brew will prompt you to add the following to your pythonpath in .bash_profile:

export PYTHONPATH=/usr/local/lib/python:$PYTHONPATH

Continue installing:

brew install zmq
pip install pyzmq
pip install pygments

And that's it -- you should now be able to launch the qtconsole by executing ipython qtconsole. If you want to see the matplotlib output inline (and why wouldn't you?), then execute:

ipython qtconsole --pylab=inline

Fin.

Phew! I hope this works for everyone out there. Until the next software upgrade, anyway.

Updated 9/1/11 to include virtualenv.

Tagged as: code, ipython, matplotlib, numpy, python, scipy