Recently I've been on a bit of a "turn stuff into Markdown files and slap them in an Obsidian Vault" trip. This kicked off a couple of months back when I made a decision unrelated to coding.

On and off, since my teenage years, I've kept journals. Since those teenage years it's been more off than on, but a couple of times in my adult life it's been really helpful to actually write one. The last time this happened was early 2019. It was pretty vital I did that at the time and it was a really sensible and helpful decision, and an approach to the situation I was in that I'd recommend to anyone (and have done on occasion to anyone going through the same thing).

The actual motivation for starting that particular journal is long behind me, but I'd got into the habit of writing it and so, until a couple or so months back, I kept jotting something down every day. But I came to the realisation that I didn't need to and that it had become something of a chore.

I'd been using an application called Journey. It's a great app, does the job well, but was also suffering from the creep of "AI" (I've had a few apps ion my arsenal that don't need it, acquire a useless "AI" feature). This privacy-problematic change of direction, combined with the realisation that I didn't need to write about my day, every day, any more, made me decide it was time to stop and cancel the subscription.

Thankfully Journey has a pretty comprehensive export option so I used it and didn't think too much more about it for a while.

Meanwhile I also had a subscription to Evernote that I didn't really use any more. Within it I had held a handful of years of journal entries from a decade or so ago, along with other "remember this for some point in the future" stuff. For the longest time I was on some really cheap tier that didn't exist any more, one that was low enough that I didn't really notice the cost go out each month so I kept putting off exporting things and closing it all down until "next month".

Then I got an email from them to say they were forcing me onto some new tier that was more expensive. So that was the final straw there. I made an export of what I had in Evernote and closed that account down too.

A wee while went past and then I got to thinking that it might be interesting to try and combine both these sources into one archived journal. I had stuff from around 2010 to 2015, and I also had stuff from 2019 until 2024; the former in the Evernote archive and the latter in the Journey archive. Surely I could write a couple of tools to turn that data into one consolidated Obsidian Vault?

Over the course of a couple of weekends journey2md and evernote2md were born. While both of those tools work differently, they're both designed to populate the same Obsidian Vault. Once I was happy with this I did the mass conversion and I was happy with the result.

Now I have years of journal entries, all converted to Markdown files and made available for reading via an application that lets me rummage through history using dates and tags and all sorts of other searching.

So I was happy with that and didn't give it much more though.

Then last week I got to thinking...

Twitter has turned into the worst place possible and I can't for the life of me think why any right-thinking person who has an ounce of humanity or has anything approaching a humanistic outlook on life would remain an active user. Honestly I stuck it out longer than was sensible, but in June 2023 I finally quit for good.

Back when the new owner was confirmed I, like a lot of people, extracted my archive. It's since been sat in storage doing nothing, yet there's a lot of data in there that could be interesting to work with, or just to go back and look through. So last week's thought was "why don't I also turn this into an Obsidian Vault?".

So I did...

The tool I built to do this is bird2glass. As you'll see in the README it makes a few assumptions about the state of Twitter archive dumps and also what a user wants from this. Personally I'm pleased with the result.

The main aim of the tool is to break the tweets down into a hierarchy of year, month and day...

...and also to connect them with any account that was being replied to or mentioned in some way...

This user view is handy when viewing backlinks, as it gives you a list of all the tweets that mention that user (and, of course, if you're into Obsidian's graph it will make for some interesting connections within there).

I sense there's more I can do with this, and I imagine I will continue to tinker with it. Meanwhile though, if that sounds like something you'd benefit from do feel free to grab it and play with it and hack on it. Keep in mind the notes and assumptions that are in the README, and really be prepared for a lot of files to be created if you did a lot of tweeting like I did (I do think that over 50,000 individual files for an Obsidian Vault is a bit silly, if I'm honest).

Meanwhile... I might need to look at other applications and think about how I can turn the data into useful Markdown collections!

I was quite late discovering Pinboard; by the looks of things I created my account and paid my first subscription for it in early 2019. Since then I've been a pretty avid user and found it really useful. I've even written a couple of clients for it (one for Emacs and one for the terminal).

During that time it's had its fair share of hiccups and outages, but on the whole I've found it a stable and useful service.

The service does have its detractors, and concerns over its long-term stability and how well it's maintained are fairly common. I half paid attention to these, and had started to think about where I might go if there was an issue.

While maintaining and syncing bookmarks isn't exactly a difficult or unsolved problem, and while it's also true that it could be fun to roll my own solution, there are a couple of things I need that would make building my own approach a bit of a chore.

Things important to me are:

• An extension for any random browser I might find myself using
• A good mobile client for at least iOS and iPadOS
• A good API so I can write my own tools if I need to
• A clean and focused backend website

I kept these things in mind and kept an eye out but I'd never really felt the need to actively start looking around.

Then I stumbled on this after posting about another Pinboard outage.

That... yeah, that was the final push I needed to start to think seriously about where to move and how.

A couple of people suggested Raindrop, and from what I could tell it was coming up as a pretty popular service that some Pinboard users were migrating to. I had a look and it wasn't quite what I was after; but close.

You see, there's two things I really like about Pinboard that Raindop didn't seem to cover:

• Simple support for "this shit is unread". I see things, I share to whatever Pinboard app I have on my phone or tablet, etc, and then I review some time later (normally in Tinboard).
• Support for Private and Public pins. I've liked having a feed of bookmarks I can let people see and Raindrop doesn't have this.

I looked around at some blogs that talked about Pinboard vs Raindrop and didn't see any that really dived into this particular aspect of migrating; I also asked a couple of folk who'd made the move about this and they didn't really have any insight (mainly because they didn't care about those particular uses).

One thing I did notice though was that Raindrop does support making individual collections public. So, if I was willing to sacrifice any other uses for collections (a bookmark in Raindrop can only be in one collection), I could simply have a Public and a Private collection and import pins into the appropriate one. Also, unread pins could be left out of the collections and I could use that to signify unread status.

This seemed fine as I'm heavy on the tags anyway.

Now... Raindrop has a pretty comprehensive import facility built in. I gave it a try with Pinboard's backup file and it worked really well. That is... really well except it just threw away the public/private/unread aspect of the pins. There was only one thing for it then: I had to write my own importer!

Which brings me to Paindrop. It's a quick hack but it does the job, and it does the import just how I wanted. The result of the first test was pretty much spot on (in this image I'm comparing what Raindrop says vs what Tinboard says I have in Pinboard):

Usage is pretty straightforward. You create Public and Private collections in Raindrop, you create an app in Raindrop and get the access token, you grab your Pinboard access token and then:

$ paindrop example:xxxxxxxxxxxxxxxxxxxx xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

where the first parameter is the Pinboard access token and the second the Raindrop access token.

If all goes well, after a few moments, the importer should finish and you should find that all of your pins have migrated to Raindrop, all public pins are in the Public collection and all private pins are in the Private collection. Any pins that were marked as unread will be Unsorted.

Note that if you used different names for your public and private collections you can pass those names to paindrop with the --public and --private switches.

If you're looking to move your bookmarking history out of Pinboard and want to keep the same sort of structure I had I hope Paindrop will be useful to you too.

Paindrop can be installed with pip or (ideally) pipx from PyPi. It can also be installed with Homebrew by tapping davep/homebrew and then installing paindrop:

$ brew tap davep/homebrew
$ brew install paindrop

The source is available on GitHub.

PS: As for the name... originally it was pin2rain but then Darren Burns pointed out the obvious and it had to happen.

I've just released Tinboard v0.14.0. This release adds a feature that a user requested, where you can set the default values for the privacy and read-later status of a new bookmark:

So, any time you create a new bookmark, the edit dialog will use those values by default. It's a feature that makes perfect sense but I didn't think to add it early on because... well, I set the defaults to my preference.

Tinboard can be installed with pip or (ideally) pipx from PyPi. It can also be installed with Homebrew by tapping davep/homebrew and then installing tinboard:

$ brew tap davep/homebrew
$ brew install tinboard

The source is available on GitHub.

Tinboard has turned into a tool I use pretty much every day; it's probably my most-used Textual/Python-developed application at this point. This is causing me to think more and more about how I can add things to it that are related to the core purpose, but are also outside of the main "interface with Pinboard" thing.

A thing with keeping bookmarks for a long time is that some of them go stale, go away. Some will just plain 404, others the whole site will disappear. If I find myself going back to a bookmark and seeing this is the case, I'll hit the Wayback Machine and see if there's an archive there.

So I got to thinking: what if I add the ability to perform this check into Tinboard itself? So I did just that.

Now, in the application, if you press w with a bookmark highlighted, it will check with the Wayback Machine to see if the bookmark is in the archive. If it isn't you see this:

On the other hand, if it is in the archive, you'll see something like this:

I sense this is going to be the first step in a couple of features related to this. I'm thinking that I may go on to add a "swap the URL for this bookmark with the Wayback Machine archive URL" feature, which will be handy for those bookmarks that have one away, and it would also be useful to look at the options for a "please archive a copy of this bookmark" feature.

But, for now, v0.12.0 is available and has this handy (for me anyway) first step.

Tinboard can be installed with pip or (ideally) pipx from PyPi. It can also be installed with Homebrew by tapping davep/homebrew and then installing tinboard:

$ brew tap davep/homebrew
$ brew install tinboard

The source is available on GitHub.

Back in the very early days of the Textual adventure, within the first month or so of working on the framework, we had a period of dogfooding. One of the projects I wrote during that time was a little tool I called PISpy.

The initial version was pretty much a quick hack; during that dogfooding period I did my best to try and develop a new project every couple of days. Since then I've let PISpy descend into bit rot.

In the last week or so I've turned my attention back to it and made an effort to tidy up the code, tidy it some more, and some more, and even some more.

This morning I put the finishing touches to these changes and released v0.6.0.

PISpy can be installed with pip or (ideallty) pipx from PyPI. It can also be installed with Homebrew by tapping davep/homebrew and then installing pispy:

$ brew tap davep/homebrew
$ brew install pispy

The source is available on GitHub.

While my time working on Textual might have come to an end, my time working with Textual hasn't. Three days back I experimented with Textual's newly-added "inline mode":

In doing so I extended the application so that it's possible to run tinboard add and quickly enter a new bookmark and then carry on in the terminal, without needing to "go fullscreen". I'll admit it's of limited use, but it seemed like a good shakedown of the feature and in working on it I was able to discover a couple of bugs (#4385, #4403).

The effect of this is this:

Tinboard can be installed with pip or (ideally) pipx from PyPi. It can also be installed with Homebrew by tapping davep/homebrew and then installing tinboard:

$ brew tap davep/homebrew
$ brew install tinboard

The source is available on GitHub.

While I have been on the receiving end of redundancy once before, that was after 21 years of service at a company that, while it was in part about software development, I would never have called it a "tech" company.

So, as of today, I can finally say that the "tech layoffs" came for me and I'm one of 67% of employees being let go from a tech startup.

Achievement unlocked, I guess?

To be clear: I'm not annoyed about this, I'm not even shocked about this; I planned for this from the off and realised and recognised the gamble I was taking back in 2022.

I am disappointed about this. Not in a "I'm disappointed in you" kind of way, but disappointed for all involved and what it says about how FOSS projects are funded and maintained.

It's been an interesting journey, and it's been a privilege to do something I've been wanting to do since the 1990s, when I first read the GNU Manifesto and subsequently watched the free software and open source movements develop and grow: work on FOSS for a living. In doing this I've developed my thoughts about the feasibility of such an endeavour, I've refined how I feel about working in very small teams, I've learnt a lot of useful lessons I'm going to draw on in the future (keeping a journal of my experience has been a great move; I have a lot of notes and thoughts written down that I'll be reviewing and distilling for myself over the coming weeks).

Most of all: it's been an absolute blast working on something that people are actually using to build cool things, and to provide help and guidance to those people when they've needed it.

So... what happens now? Well, of course, right now, I'm looking for a new position. If you're reading this and you are looking for someone who's kinda handy with Python and a bunch of other languages and who loves learning new stuff, or if you know someone who is looking for such a person, do drop me a line!

As for what happens with Textual, and my involvement with it...

Well, what happens with Textual is Will's call, of course. As for my involvement with it: I care about FOSS and I care about Textual; I also care about the folk who have been kind enough to use their time to explore it, test it, build with it, commit to it and make neat stuff with it. My intention, as long as free time allows, is to carry on being involved, both on GitHub and in the Discord server.

It's my sincere hope that, as a community of FOSS-friendly developers, we see Textual over the 1.0 line and beyond.

But all that starts next week. It's a bank holiday weekend and I think I might have deserved a run, a bit of mucking about in VR, a beer, and just a wee bit of down time.

Over the past year and a half I've written a lot of Python code, and a lot of that Python code has been Textual applications; most of those Textual applications have been very quick demonstration or test applications built to help support someone asking for help; some of them have been less-trivial applications written in my own time and for my own use and amusement. Of them I'd say there are two near-daily-drivers, and a couple that I either have more plans for, or like to maintain just for the hell of it.

Those latter applications are all ones that I've deployed to PyPI, and because of that are all ones that I've recommenced be installed using pipx. During that time though I've had half an inclination to make them installable via Homebrew. While probably not installable from the core Homebrew repository¹, at least installable from a "tap"² that's under my own GitHub account or something.

To this end I've had a blog post about packaging Python apps for Homebrew saved in Pinboard for a while now, and every time I look at it I think "this is a lot of faff, maybe later". Today was that "later".

As it turned out, it was way easier than I first realised. The evolution of today pretty much went like this:

Deciding to use a single repository as the "tap"

The blog post above seemed to suggest that for every application repository you want a tap for, you probably want a parallel homebrew--prefixed repository. This in turn would suggest that every time someone wants to install one of your tools, they'd need to add a new tap³. As I looked at it this seemed like way too much faff, so in the end I decided to create a single repository that I'd keep all my formula files in. The naming of homebrew-homebrew meant that the tap name would simply be davep/homebrew.

Simple and clean, I think: things for homebrew, things that can be installed via homebrew, that come from davep. To add the tap it's simply:

$ brew tap davep/homebrew

Ensuring that all my applications and libraries publish source

Although it seems that it might be (possibly, maybe, perhaps, who can tell?) deprecated, it looked like homebrew-pypi-poet was a tool I'd need to do all the heavy work on making the formula file. A quick test threw up a problem where it was complaining that my test package (one of my own applications) didn't have an sdist. Sure enough, through nothing more than never having bothered to make it happen, the source of my libraries and applications wasn't been uploaded to PyPI when I published.

So I went through some of my repositories and fixed that, making patch releases as I went.

Making a Makefile to let me be lazy

The next thing to do was to figure out the most lazy way of building the formula files. From what I could see the main steps to making all of this work were:

• Make a venv and activate it
• Install homebrew-pypi-poet
• Install the package you want to package for Homebrew
• Run poet to make the formula

Seemed simple enough. For all sorts of lazy reasons I still tend to use pipenv to get things done quickly, and that seemed to work fine here too. I'm also a fan of PIPENV_VENV_IN_PROJECT=true which makes things clean and tidy, so I figured a rule in a Makefile like this:

clean:
        rm -rf .venv
        rm -f Pipfile Pipfile.lock
        pipenv --python 3.12
        pipenv install --dev homebrew-pypi-poet

would be fine to make a clean venv ready to build the formula, and then I'd have a rule for the package itself that depended on the above, like this:

oshit: clean
        pipenv install oshit
        pipenv run poet -f oshit > Formula/oshit.rb

Fixing the package description

The above was great, and worked really well. But there was one issue that I could see: the resulting formula file always had this desc inside it:

desc "Shiny new formula"

From what I could see there was no way to tell poet what I wanted the description to be, and neither did I want to have to remember to edit that line each time I regenerated the formula file. So sed to the rescue then I guess, with this sort of thing:

sed -i '' 's/Shiny new formula/The actual text I want/' Formula/coolapp.rb

The result

The result of all of this is that I now have a repository that I or anyone else can use as a tap to be able to install my stuff using the brew command. So now if you want a little Hacker News reader for the terminal but you don't want to be messing with installing pipx and the like, but you do use brew on your machine, it's just this:

$ brew tap davep/homebrew
$ brew install oshit

Fingers crossed it all "just works" when I next upgrade one of those packages. I will, of course, have to remember to go into davep/homebrew-homebrew and make the-app for the relevant application, and then commit and push the changes, but that's really not too difficult to remember and do.

Hopefully it'll then all just work.

I just realised that it's been a while since I last posted an update about tinboard. This is probably my most-used Textual-based application, and one I'm constantly tinkering with, and just this morning I published v0.10.0.

Often the changes are small tweaks or fixes to how it works, sometimes they're simply updates to the version of Textual used, making use of some new feature or other; I've yet to add another "major" feature so far. They will come, but so far the ideas I have for the application haven't actually felt that necessary. Although I say so myself it does what I need it to do and it does it really well.

So, as a quick catch-up of what's changed since v0.4.0 (which was the last version I posted about):

• v0.5.0 was released 2024-01-04; this included all the tags of a bookmark when doing full-text searching.
• v0.6.0 was released 2024-01-10; it fixed a small bug where the tag suggestion facility got confused by trailing spaces in the input field.
• v0.7.0 was released 2024-02-02; this updated the minimum Textual version to v0.48.2 and removed all the custom changes to the Textual TextArea widget, making use of the updates to TextArea that version of Textual made available.
• v0.8.0 was released 2024-02-18; this fixed a crash on startup caused by a newer release of Textual (the fault was in tinboard; the update to Textual helped reveal the problem).
• v0.9.0 was released 2024-02-29; it simply added support for using Esc at the top level of the application to quit (I like to camp on Esc to GTFO).

Then, just now, I released v0.10.0. This release makes full use of some work I recently did to enhance Textual's CommandPalette widget, which added a "discover" system. The change in tinboard is that all of the command palette providers now have discover methods too. The result of this change is that when you open the command palette in tinboard (ctrl+p) you can see every possible command right away.

Tinboard can be installed with pip or (ideally) pipx from PyPi. The source is available on GitHub.

I feel like I'm on a bit of a roll when it comes to building applications for the terminal at the moment; while I'm still tinkering and improving tinboard and OSHit, I had the urge to tackle another idea that's been on my TODO list for a while.

This is something I did for Emacs back in 2017 and I felt it was a perfect candidate for a Textual-based project. It's a terminal-based trivia quiz game, using the Open Trivia Database as the source of questions.

I've just published an early version to PyPI; it still needs some polish and I have a few other ideas for it, but as it stands I feel it's a fun little game to mess around with.

The idea is pretty straightforward: you can run it up and create lots of different quizzes, there are various parameters you can use to create lots of different kinds of challenges:

Once you're created a quiz, you can run it and answer away:

Right now the idea is that you answer by pressing either 1, 2, 3 or 4 (or just 1 or 2 for true/false questions); when I get a moment I'll also enable mouse support for selecting an answer too (honestly I feel keyboard-answering feels far more natural).

Once the quiz is done you can review your answers and see which were right and which were wrong:

As I say: there's a bunch of other things I want to add to this (keeping track of scores, adding session token support to reduce the chances of repeat questions, etc), but this felt like a good spot to make a v0.1.0 available if anyone else wanted to have a play.

Anyway, if this sounds like your sort of thing, it can be installed with pip or (ideally) pipx from PyPi. The source is available over on GitHub.

PS: Now you can see why I made textual-countdown.