Python and macOS
On Twitter, a few weeks back, @itsBexli asked me how I go about setting up Python for development on macOS. It’s a great question and one that seems to crop up in various places, and since I first got into using macOS and then subsequently got back into coding lots in Python it’s absolutely an issue I ran into.
With my previous employer, while I was the only developer, I wasn’t the only person writing code and more than one other person had this issue so I eventually wrote up my approach to solving this problem. That document is on their internal GitLab, but I thought it worth me writing my personal macOS/Python “rules” up again, just in case they’re useful to anyone else.
I am, of course, not the first person to tackle this, to document this, to write down a good approach to this. Before and after I settled on my approach I’d seen other people write about this. So… this post isn’t here to try and replace those, it’s simply to write down my own approach, so if anyone asks again I can point them here. I feel it’s important to stress: this isn’t the only way or thoughts on this issue, there are lots of others. Do go read them too and then settle on an approach that works for you.
One other point to note, which may or may not make a difference (and may or
may not affect how this changes with time): for the past few years I’ve been
a heavy user of
pipenv to manage my
virtual environments. This is very likely to change from now on, but keep in
mind that what follows was arrived at from the perspective of a
So… with that admin aside…
When I first got back into writing Python it was on macOS and, really early on, I ran into all the usual issues: virtual environments breaking because they were based on the system Python and it got updated, virtual environments based on the Homebrew-installed Python and it got updated, etc… Simply put, an occasional, annoying, non-show-stopping breaking of my development environment which would distract me when I’d sat down to just hack on some code, not do system admin!
For me, what’s worked for me without a problem over the past few years, in short, is this:
- NEVER use the system version of Python. Just don’t.
- NEVER use the Homebrew’s own version of Python (I’m not even sure this is an issue any more; but it used to be).
- NEVER use a version of Python installed with Homebrew (or, more to the point, never install Python with Homebrew).
- Manage everything with
pyenv; which I do install from Homebrew.
As mentioned earlier, what I’m writing here assumes that virtual
environments are being managed with
pipenv (something I still do for
personal projects, for now, but this may change soon). Your choices and
mileage may vary, etc… This is what works well for me.
The “one time” items
These are the items that need initially installing into a new macOS machine:
Unless it comes from the Mac App Store, I try and install everything via Homebrew. It’s really handy for keeping track of what I’ve got installed, for recreating a development environment in general, and for keeping things up to date.
With Homebrew installed the next step for me is to install
pyenv. Doing so
is as easy as:
$ brew install pyenv
Once installed, if it’s not done it for you, you may need to make some changes to your login profile. I’m a user of fish so I have these lines in my setup. Simply put: it asks pyenv to set up my environment so my calls to Python go via its setup.
Plenty of help on how to set up
pyenv can be found in its
Once I’ve done this I tend to go on and install the Python versions I’m likely to need. For me this tends to be the most recent “active” stable versions (as of the time of writing, 3.7 through 3.10; although I need to now start throwing 3.11 into the mix too).
I use this command:
$ pyenv install --list
to see the available versions. If I want to see what’s available for a specific version I’ll pipe through grep:
$ pyenv install --list | fgrep " 3.9"
This is handy if I want to check what the very latest release of a specific version of Python is.
The “Global” Python
When I’m done with the above I then tend to use
pyenv to set my “global”
Python. This is the version I want to get when I run
python and I’m not
inside a virtual environment. Doing that is as simple as:
$ pyenv global 3.10.7
Of course, you’d swap the version for whatever works for you.
When Stuff Breaks
It seems more rare these days, but on occasion I’ve had it such that some update somewhere still causes my environment to break. Now though I find that all it takes is a quick:
$ pyenv rehash
and everything is good again.
Setting Up A Repo
With all of the stuff above being mostly a one-off (or at least something I
do once when I set up a new machine – which isn’t often), the real “work”
here is when I set up a fresh repository when I start a new project. Really
it’s no work at all. Again, as I’ve said before, I’ve tended to use
for my own work, and still do for personal stuff (but do want to change
that), mileage may vary here depending on tool.
When I start a new project I think about which Python version I want to be
working with, I ensure I have the latest version of it installed with
pyenv, and then ask
pipenv to create a new virtual environment with
$ pipenv --python 3.10.7
When you do this, you should see
pipenv pulling the version of Python from
$ pipenv --python 3.10.7 Creating a virtualenv for this project... Pipfile: /Users/davep/temp/cool-project/Pipfile Using /Users/davep/.pyenv/versions/3.10.7/bin/python3 (3.10.7) to create virtualenv... ⠙ Creating virtual environment...created virtual environment CPython3.10.7.final.0-64 in 795ms creator CPython3Posix(dest=/Users/davep/temp/cool-project/.venv, clear=False, no_vcs_ignore=False, global=False) seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/davep/Library/Application Support/virtualenv) added seed packages: pip==22.2.2, setuptools==65.3.0, wheel==0.37.1 activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator ✔ Successfully created virtual environment! Virtualenv location: /Users/davep/temp/cool-project/.venv Creating a Pipfile for this project...
The key thing here is seeing that
pipenv is pulling Python from
~/.pyenv/versions/. If it isn’t there’s a good chance you have a Python
earlier in your
PATH than the
pyenv one – something you generally don’t
want. It will work, but it’s more likely to break at some point in the
future. That’s the key thing I look for; if I see that I know things are
going to be okay.
Since I adopted these personal rules and approaches (and really, calling them “rules” is kind of grand – there’s almost nothing to this) I’ve found I’ve had near-zero issues with the stability of my Python virtual environments (and what issues I have run into tend to be trivial and of my own doing).
As I said at the start: there are, of course, other approaches to this, but this is mine and works well for me. Do feel free to comment with your own, I’m always happy to learn about new ways!