Over the past year and a half I've written a lot of Python code, and a lot of that Python code has been Textual applications; most of those Textual applications have been very quick demonstration or test applications built to help support someone asking for help; some of them have been less-trivial applications written in my own time and for my own use and amusement. Of them I'd say there are two near-daily-drivers, and a couple that I either have more plans for, or like to maintain just for the hell of it.

Those latter applications are all ones that I've deployed to PyPI, and because of that are all ones that I've recommenced be installed using pipx. During that time though I've had half an inclination to make them installable via Homebrew. While probably not installable from the core Homebrew repository¹, at least installable from a "tap"² that's under my own GitHub account or something.

To this end I've had a blog post about packaging Python apps for Homebrew saved in Pinboard for a while now, and every time I look at it I think "this is a lot of faff, maybe later". Today was that "later".

As it turned out, it was way easier than I first realised. The evolution of today pretty much went like this:

Deciding to use a single repository as the "tap"

The blog post above seemed to suggest that for every application repository you want a tap for, you probably want a parallel homebrew--prefixed repository. This in turn would suggest that every time someone wants to install one of your tools, they'd need to add a new tap³. As I looked at it this seemed like way too much faff, so in the end I decided to create a single repository that I'd keep all my formula files in. The naming of homebrew-homebrew meant that the tap name would simply be davep/homebrew.

Simple and clean, I think: things for homebrew, things that can be installed via homebrew, that come from davep. To add the tap it's simply:

$ brew tap davep/homebrew

Ensuring that all my applications and libraries publish source

Although it seems that it might be (possibly, maybe, perhaps, who can tell?) deprecated, it looked like homebrew-pypi-poet was a tool I'd need to do all the heavy work on making the formula file. A quick test threw up a problem where it was complaining that my test package (one of my own applications) didn't have an sdist. Sure enough, through nothing more than never having bothered to make it happen, the source of my libraries and applications wasn't been uploaded to PyPI when I published.

So I went through some of my repositories and fixed that, making patch releases as I went.

Making a Makefile to let me be lazy

The next thing to do was to figure out the most lazy way of building the formula files. From what I could see the main steps to making all of this work were:

• Make a venv and activate it
• Install homebrew-pypi-poet
• Install the package you want to package for Homebrew
• Run poet to make the formula

Seemed simple enough. For all sorts of lazy reasons I still tend to use pipenv to get things done quickly, and that seemed to work fine here too. I'm also a fan of PIPENV_VENV_IN_PROJECT=true which makes things clean and tidy, so I figured a rule in a Makefile like this:

clean:
        rm -rf .venv
        rm -f Pipfile Pipfile.lock
        pipenv --python 3.12
        pipenv install --dev homebrew-pypi-poet

would be fine to make a clean venv ready to build the formula, and then I'd have a rule for the package itself that depended on the above, like this:

oshit: clean
        pipenv install oshit
        pipenv run poet -f oshit > Formula/oshit.rb

Fixing the package description

The above was great, and worked really well. But there was one issue that I could see: the resulting formula file always had this desc inside it:

desc "Shiny new formula"

From what I could see there was no way to tell poet what I wanted the description to be, and neither did I want to have to remember to edit that line each time I regenerated the formula file. So sed to the rescue then I guess, with this sort of thing:

sed -i '' 's/Shiny new formula/The actual text I want/' Formula/coolapp.rb

The result

The result of all of this is that I now have a repository that I or anyone else can use as a tap to be able to install my stuff using the brew command. So now if you want a little Hacker News reader for the terminal but you don't want to be messing with installing pipx and the like, but you do use brew on your machine, it's just this:

$ brew tap davep/homebrew
$ brew install oshit

Fingers crossed it all "just works" when I next upgrade one of those packages. I will, of course, have to remember to go into davep/homebrew-homebrew and make the-app for the relevant application, and then commit and push the changes, but that's really not too difficult to remember and do.

Hopefully it'll then all just work.

Earlier this week something in my development environment, relating to Homebrew, Python, pyenv and pipenv, got updated and broke a handful of repositories. Not in a way that I couldn't recover from, just in a way that was annoying, got in the way of my workflow, and needed attention. (note to self: how I set up for Python/Django development on a machine might be a good post in the future)

Once I was sure what the fix was (pretty much: nuke the virtual environment and recreate it with pipenv, being very explicit about the version of Python to use) the next step was to figure out how many repositories were affected; not all were and there wasn't an obvious pattern to it. What was obvious was that the problem came down to python in the .venv directory pointing to a binary that didn't exist any more.

So... tracking down problematic repositories would be simple enough, just look for every instance of .venv/bin/python and be sure it points to something rather than nothing; if it points to nothing I need to remake the virtual environment.

I quickly knocked up a script that was based around looking over the results of a find, and initially decided to use file to perform the test on python. It seemed to make sense, as I wrote the script I checked the man page for file(1) on macOS and sure enough, this exists:

-E On filesystem errors (file not found etc), instead of handling the error as regular output as POSIX mandates and keep going, issue an error message and exit.

Given that file dereferences links by default, that should get me an error for a broken link, right? Bit hacky I guess, but it was the first thing that came to mind for a quick bit of scripting and would do the trick. Only...

$ file -E does-not-exist
file: invalid option -- E
Usage: file [bcCdEhikLlNnprsvzZ0] [-e test] [-f namefile] [-F separator] [-m magicfiles] [-M magicfiles] file...
       file -C -m magicfiles
Try `file --help' for more information.

Wat?!? But it's right there! It says so in the manual! -E is documented right in the manual page! And yet it's not in the valid switch list as put out by the command, and it's an invalid option. The hell?

So I go back and look at the man page again and then I notice it isn't in the list of switches in the synopsis.

SYNOPSIS
file [-bcdDhiIkLnNprsvz] [--extension] [--mime-encoding] [--mime-type] [-f namefile] [-m magicfiles] [-P name=value] [-M magicfiles] file
file -C [-m magicfiles]
file [--help]

I then did the obvious tests. Did I have file aliased in some way? No. Was some other thing that works and acts like file in my path? No. Was I absolutely 100% using /usr/bin/file? Yes.

Long story short: it seems the man page for file, on macOS, fibs about what switches it supports; it says that -E is a valid option, but it's not there.

What's even odder is the man page says it documents v5.04 of the command, but --version reports v5.37. Meanwhile, if I check on a GNU/Linux box I have access to, it does support -E, reports it in the switches, documents it in the man page (in both the synopsis and in the main body of the page) and it is v5.25 (and so is its man page).

So that was something like 20 minutes lost to a very small problem, for which there was no real solution, but was time that had to be spent to get to the bottom of it.

In the end I went with what I probably should have gone with in the first place: stat -L.

for venv in $(find . -name .venv)
do
    if ! stat -L "$venv/bin/python" > /dev/null 2>&1
    then
        echo "$(dirname $venv)"
    fi
done

And now I have that script in my ~/bin directory, ready for the next time Homebrew and friends conspire to throw my day off for a while.

Earlier on today I needed a copy of wget on my iMac. It's not "native" to it so I got to wondering how you go about getting something like that onto it. Sure, I could have just grabbed the source and built myself, but really it's a lot nicer to use some sort of package manager.

A quick search lead me to Homebrew and I was then up and running in no time.

This in turn got me to thinking about how it might be fun to get some of the software I used to use on my GNU/Linux machine up and running again. The first one that came to mind was slrn. Sure enough slrn is available via Homebrew and installing it was dead simple.

But then I was faced with a problem: I needed an NNTP server. Way back I used to run a local one in my office that fed from and to my ISP's. Back then my ISP was Demon Internet; these days I'm with BT. A quick search lead me to an article or two that BT had a NNTP server, of sorts, provided by a third party. So I did a quick check:

Yay! This looked good.

After that I fired up slrn and.... problems. It kept asking me to log in, to provide a user name and password. The only problem was that I'd read in more than one place that a user name and password weren't needed for BT's server; all that was required was you be on a BT IP address. Checking the slrn docs I found force_authentication but ensuring that was off made no difference.

At this point I removed slrn and gave up.

Later, thinking it might be an issue with just slrn and perhaps it was worth trying a native NNTP client, I grabbed Unison (which is no longer supported but seems to work fine). I got that set up and ran into the same issue: it wanted login details.

Finally, after a bit more digging, I stumbled on the reason why I was struggling to make any of this work: BT had closed support for the server back in December last year!

A quick search around the web and I stumbled on Eternal September. Given all I was interested in was the good old text groups this looked perfect. I quickly registered an account, ran up Unison again and plugged in my details and....

Now that's all sorted I should try again with slrn. At which point I'll need to drag out and tidy up post.el (the version that was being maintained by some other people seems to have gone very stale, sadly).