08 Jan Python — Managing custom installations in OS X
Update
Since this post was first written, I’ve discovered some people use Anaconda Python. After investigating, it looks like Anaconda is a much simpler solution. To deploy it, I installed both Anaconda 2 and 3 to /usr/local/python/ , and then modified the /etc/paths and /etc/manpaths files accordingly (as shown further down in the article).
Overview
This post walks you through the steps to create new installations of Python on an OS X machine you administrate. The end-goal is to have standalone versions of Python 2 and 3 that you can manage easily. These Python installations will take precedence in your users’ executable paths. This means that when a user types python or python3 , they will be using your customized Python installations instead of the system default Python or some other version.
Note that this walkthrough is aimed at systems administrators who want more control over which versions of Python are being deployed to their OS X machines.
Recently, I’ve started restructuring how we manage custom Python installations. This came about because of a few factors, specifically: (1) our users often request additional libraries be installed that require administrative access, and (2) I like to keep the most up-to-date Python executables available for use. The OS X baked-in Python is not easily modifiable or updatable, so I investigated alternative methods of installing Python and managing its libraries. Previously, we just used the installers provided at Python’s website, but these are not easy to maintain or update from an administrative standpoint. I wanted something that included very explicit but easy-to-guess paths, among other functionality. Initially I tried installing Python using Homebrew. However, there were some problems with that method due to a combination of the way we deploy software and the way Homebrew builds the Python executables. So the final — and currently best — solution was to use a tool called pyenv to do the building.
pyenv
pyenv is generally designed to be used on a per-user basis. It allows each user to install their own Python versions separately, and then the executables for these are pushed to the front of a user’s path. This ensures that they are the versions used when the user types python on the command line. pyenv also makes it easy to switch default Python versions, and you can even specify a different Python version for a specific directory.
However, the thing I liked most about pyenv is it provides an additional tool called python-build . The python-build tool makes it easy to install specific versions of Python into any location you want, with each version’s dependencies existing only within the install location for that version. This means that each Python version is totally self-contained and can’t interfere with other versions.
Installing pyenv
I installed pyenv for the root user and for our template user profile. Installing pyenv is pretty straightforward; simply do the following to install for the current user:
$ curl -L https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash $ export PATH="~/.pyenv/bin:$PATH" $ eval "$(pyenv init -)" $ eval "$(pyenv virtualenv-init -)"
Note: The above example requires your current shell to be Bash to work properly. If your shell is not Bash, you’ll need to look up how to set environment variables in your shell.
Installing Python
I decided to install each Python version (2.7.11 and 3.5.1) into their own explicitly-named directories in /usr/local/python/ . To install Python 2 system-wide, do the following as root: $ python-build 2.7.11 /usr/local/python/2.7.11 . This will create a /usr/local/python/2.7.11 directory which will contain all of the necessary files for your Python 2 installation.
Unfortunately, /usr/local/python/ is not part of the default PATH (the variable that describes where all your executable files are located). This means that your users won’t be able to use any of the executables for your new Python version unless they use absolute paths… which is fairly annoying. What we want is for these new Python executables to be (1) easily usable without absolute paths (i.e. $ python instead of $ /usr/local/python/2.7.11/bin/python ); and (2) to appear ahead of other Python binaries, such as the default OS X /usr/bin/python .
Modifying the Path
OS X provides a tool called path_helper to build the default user path, and it builds the values from a few locations. (The executable is located at /usr/libexec/path_helper .) We can modify the path built by path_helper by editing the file at /etc/paths or adding files to /etc/paths.d/ . You can also add paths to manual pages by editing the file at /etc/manpaths or the files in /etc/manpaths.d/.
However, I wanted to make this system easy to upgrade in the future, too. The simplest way I found of doing this was to create a symbolic link in the /usr/local/python directory that points to the most recent version of Python 2 or 3 that you want to use. So, for this example, I did $ ln -s /usr/local/python/2.7.11 /usr/local/python/2-current so that the executables I want to access will be located in /usr/local/python/2-current/bin/. This means that when I upgrade things in the future, I will only have to update the symbolic links to ensure everything stays connected.
Now there is a choice: either modify the /etc/paths file, or add a file into the /etc/paths.d/ directory. If there were no other versions of Python on the system, it really wouldn’t matter which you choose. However, I want to guarantee that the new version I’ve installed with python-build is the first version of Python that is found. path_helper loads the paths from /etc/paths first, and then appends the paths found in the files in /etc/paths.d/ in the order in which the files are found (i.e. sort-of alphabetically). So we modify the /etc/paths file and stick our path at the top. Mine looks like this now:
/usr/local/python/2-current/bin /usr/local/bin /usr/bin /bin /usr/sbin /sbin
Note: Each line in the example above represents a path to a directory containing executables.
When a shell is next opened up, OS X will populate the default PATH variable with our new /usr/local/python/2-current/bin directory at the front, which means that users will be using the new Python by default.
Upgrading
When you inevitably decide to upgrade, simply install a new Python version to /usr/local/python/$version and update the symbolic link. You won’t have to update your path or anything else again.
I also installed Python 3.5.1 using a similar method, and so far it’s been much simpler to maintain and upgrade than our previous solutions.
No Comments