modern-ops: Python on OS-X

modern-ops requires a lot of automation.  There are too many moving parts to operate at cloud scale without tools. In the old days of a few persistent Unix servers, it was possible to get by with shell scripts but those days are gone.  Python is a great language to move to, I never really got on with Perl.

The only downside to Python is that it’s easy to get into a mess of conflicting packages and versions.

Of course there’s a solution, and the one I have chosen is pyenv + pyenv-virtualenv.  This solution was suggested by George Dowding – who is my personal Python guru.

Basically pyenv – allows multiple versions of python to co-exist on the same computer  (e.g. 2.7, 3.4, 3.6) and virtualenv, allows each Python application to have its own dependencies – e.g. specific versions of libraries, even within the same Python version family.

I am using Mac OS-X to run python, so I need to manage that too.  I am using “Homebrew” package manager.

Generally: Anything that is installed on OS-X from Disk Images or pip-install will use the standard OS-X Python.  Anything installed from git or via brew, will use pyenv.

To get all this in place we do the following. In this example, I am installing the aws cli package, that requires Python 3.4.

  1.  Install Homebrew via usual OS-X install process .
    • $ brew update
    • $ brew install pyenv
  2. Install virtualenv as part of pyenv
    • $ brew install pyenv-virtualenv
  3. Install some python versions (currently these are modern versions of Python 2.x and 3.x
    • pyenv install 3.4.9
    • pyenv install 2.7.15
  4. Create a specific virtualenv for e.g the AWS CLI package with version 3.4.9
    • In this case we call our virtual env awe-cli-pyenv-ve (-ve for virtualenv)
      • $ pyenv virtualenv 3.4.9 aws-cli-pyenv-ve
  5. Activate the virtualenv and “jump into it”.  After activation the prompt will change.
    • $ pyenv activate 3.4.9/envs/aws-cli-pyenv-ve

Note below, the prefix of the virtualenv in the shell prompt.

To get into the virtualenv from a normal shell.

Now we can run the python command.  In this case “aws”

Next we’ll setup the AWS cli so we can start manipulating the AWS environment from the CLI.