ng2web v1.0.0

Posted on 2025-03-24 14:17 +0000 in Coding • Tagged with Python, Norton Guide • 2 min read

It pretty much all started with this:

 * Revision 1.1  1996/02/15 18:57:13  davep
 * Initial revision

That's from the rcs log for the source for w3ng, a tool I write so I could read Norton Guide files in my web browser, served by Apache running on my GNU/Linux server in my office. The tool itself was written as a CGI tool (remember then?).

I believe I posted about this to comp.lang.clipper and pretty quickly some folk asked if it might be possible to do a version of that would write the whole guide as a collection of HTML files for static hosting, rather than serving them from cgi-bin utility. That seemed like a sensible idea and so:

 * Revision 1.1  1996/03/16 09:49:00  davep
 * Initial revision

ng2html was born.

Fast forward around a quarter of a decade and I decided it would be fun to write a library for Python that reads Norton Guide files, and a tool called ng2web was the first test I wrote of it, designed as a more flexible replacement for ng2html. I've tweaked and tinkered with the tool since I first created it but never actually "finished" it.

That's changed today. I've just released v1.0.0 of ng2web.

A page generated with ng2web

If turning one or more Norton Guides into static websites seems like the sort of thing you want to be doing, take a look at the documentation.

ng2web is licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

$ pipx install ng2web

It can also be installed with Homebrew by tapping davep/homebrew and then installing ng2web:

$ brew tap davep/homebrew
$ brew install ng2web

AgiNG v0.3.0

Posted on 2025-03-22 09:30 +0000 in Coding • Tagged with Python, terminal, textual, Norton Guide • 2 min read

I've just released AgiNG v0.3.0. The main focus of this release was to get some searching added to the application. Similar to what I added to WEG back in the day, I wanted three types of searching:

  • Current entry search.
  • Current guide-wide search.
  • All registered guides-wide search.

The current entry search is done with a simple modal input, and for now the searching is always case-insensitive (I was going to add a switch for this but it sort of felt unnecessary and I liked how clean the input is).

Entry search

The search is started by pressing /, and if a hit is found n will take you through all subsequent matches.

As always, if you're not sure of the keys, you'll find them in the help screen or via the command palette:

Looking up search in the command palette

Guide-wide and all-guide searching is done in the same dialog. To search guide-wide you enter what you want to find and untick "All Guides".

Setting up a guide search

With that, the search will stick to the current guide.

Guide-wide search running

As will be obvious, searching all guides that have been registered with AgiNG is as simple as ticking "All Guides". Then when you search it'll take a walk through every entry of every guide you've added to the guide directory in the application.

Gide-wide searching

Global searching is accessed with Ctrl+/ or via the command palette.

Finding global search

With this added, I think that's most of the major functionality I wanted for AgiNG. I imagine there's a few more tweaks I'll think of (for example: I think adding regex search to the global search screen could be handy), but I don't think there's any more big features it needs.

AgiNG can be installed with pip or (ideally) pipx from PyPi. It can also be installed with Homebrew by tapping davep/homebrew and then installing aging:

$ brew tap davep/homebrew
$ brew install aging

The source is available on GitHub.


AgiNG

Posted on 2025-03-12 08:00 +0000 in Coding • Tagged with Python, terminal, textual, Norton Guide • 4 min read

AgiNG

It seems I really do still have this need to create new terminal-based projects at the moment. There's been Braindrop, then Peplum, then after that came Hike. While I'm still tweaking and adding to them, and also using them to refine a wee library I'm building up that forms the core of my latest apps, I felt I still had this one app that I needed to finally build.

Since the 1990s I've had this mild obsession with building tools for maintaining access to Norton Guide files. I've written readers for GNU/Linux (which also works on macOS too), OS/2, Windows, GNU Emacs, and also or the web (in multiple incarnations). Those builds have covered a few languages, including C, C++, Pascal, JavaScript and Emacs Lisp.

I'd never written a Python library or application for it though.

So when I first saw Textual mentioned in passing on Twitter a few years back, way back in the 0.1 days, I thought that could be the thing that would push me over the edge. In anticipation of that, back in 2021, I initially developed ngdb.py. This is a library that provides the core Norton Guide reading code for Python applications and could form the basis for other tools.

As a test for this I then wrote ng2web (which works, but I think still needs a bit of tidying up -- something I'm aiming to do in the next few weeks).

Meanwhile, the journey with Textual itself kicked off, happened, and came to an end; and yet somehow I'd never got round to building the thing I'd initially looked at Textual for: a terminal-based Norton Guide reader that looked nice and modern (by terminal standards). When I initially joined Textualize the owner had actually said they wanted me to build this as test of the framework, to essentially start out by employing me to create some Free Software that would help dogfood the library, but that seemed to get forgotten.

Fast forward to the start of this month and I finally felt it was time to tackle this. The result is AgiNG1.

AgiNG in action

As of v0.1.0 it has most of the features you'd expect from a usable Norton Guide reader, including:

  • An ability to add guide files to an in-application directory.
  • The ability open and navigate a guide.
  • Full see-also support, etc.
  • Full translation of characters as were under MS-DOS into the terminal.
  • The ability to copy entry text or source to the clipboard.
  • The ability to copy save entry text or source to a file.
  • Access to a guide's credits.

AgiNG showing a long entry

I still need to write some proper documentation for the application, but meanwhile all commands and key shortcuts can be discovered either via the help screen:

AgiNG help

or by pulling up the command palette:

AgiNG command palette

Hopefully the workings of the application will be fairly obvious to anyone who is familiar with Norton Guide files; if anything isn't making sense I'm more than happy to answer questions or take suggestions for improvements.

One wee feature I want to call out, that I felt was important to add, was a "classic view" facility. The thing with Norton Guide files is they were mostly created in the very late 1980s and early-to-mid 1990s. People would often get creative with the colouring within them, but in many cases the colouring assumed the default Norton Guide application. Its colours were white text on a blue background. So sometimes other colouring was done assuming that background.

You can see an example of this here, with an entry in a guide being viewed using the default textual-dark theme:

Entry in textual-dark

Notice the colouring in the syntax section. This is more obvious if the application is switched to one of the light themes:

Entry in solarized-light

With a nod to this issue in mind, I added the "classic view" for entries (which is a sticky setting -- turn it on and it stays on until you turn it off):

Classic view in action

A little hard on the eyes, I think, but also filled with nostalgia!

Talking of themes, all the usual application themes are available, here's a wee selection:

Nord Textual Light Gruvbox Solarize Light Monokai

AgiNG is licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

$ pipx install aging

It can also be installed with Homebrew by tapping davep/homebrew and then installing aging:

$ brew tap davep/homebrew
$ brew install aging

Expect to see more updates in the near future; as with other recent projects this is very much something I'm going to be dabbling with and improving as time goes on.


  1. If you're wondering about the name, it's nothing more than a word that happens to have NG in it, and also a mild pun about this being an ageing hypertext help system; with the spelling acknowledging Peter Norton's nationality. 


MkDocs/mkdocstrings 404 CSS TIL update

Posted on 2025-02-28 22:23 +0000 in Coding • Tagged with documentation, GitHub, mkdocs, mkdocstrings, gh-pages • 1 min read

Following on from my post this morning, regarding the problem I was having with _mkdocstrings.css being 404 any time I deployed by documentation, build with mkdocs/mkdocstrings, to GitHub Pages...

It's come to light that I was doing this on hard mode, pretty much.

While trying to figure out the best way of deploying the docs, I'd stumbled on ghp-import and had been using that. On the other hand, MkDocs has it's own command for doing the same thing: mkdocs gh-deploy.

Timothée pointed out to me that he never runs into this problem, but he used this command. As it turns out, if you use mkdocs gh-deploy it creates the .nojekyll file by default.

And how does it do this? It uses the ghp-import code and uses a switch it has to achieve exactly this. In fact... the command line version even has a switch for it!

-n, --no-jekyll      Include a .nojekyll file in the branch.

This is off by default, when you run the command itself, but I wish I'd noticed this when I was first experimenting. O_o

Anyway, thanks to Timothée's pointers, I've now managed to simplify how I build and publish the docs from textual-fspicker, and I'll apply this to other projects too.


Documenting textual-fspicker (plus a TIL)

Posted on 2025-02-28 11:45 +0000 in Coding • Tagged with Python, textual, documentation, GitHub, mkdocs, mkdocstrings, gh-pages • 4 min read

I've just made a wee update to textual-fspicker, my dialog library for Textual which adds FileOpen, FileSave and SelectDirectory dialogs. There's no substantial change to the workings of the library itself, but I have added something it's been lacking for a long time: documentation!

Well... that's not quite true, it's always had documentation. I'm an avid writer of Python docstrings and I make a point of always writing them for every class, function, method or global value as I write the code. As such the low-level "API" documentation has always been sat there ready to be published somehow, eventually.

Meanwhile the description for how to use the library was mostly a pointer to some example code inside the README. Not ideal, and something I really wanted to improve at some point.

Given I'm still on a bit of a coding spree in my spare time, I finally decided to get round to using the amazing mkdocstrings, in conjunction with mkdocs, to get some better documentation up an running.

The approach I decided to take with the documentation was to have a page that gave some general information on how to use the library and then also generate low-level documentation for the all the useful content of the library from the docstrings. While latter isn't really useful to anyone wanting to use the library in their own applications, it could be useful to anyone wanting to understand how it hangs together at a lower-level, perhaps because they want to contribute to or extend the library in some way.

While writing some of the general guide took a bit of work, of course, the work to get the documentation up and running and generating was simple enough. The effort comes down to 3 rules in the Makefile for the project:

##############################################################################
# Documentation.
.PHONY: docs
docs:                    # Generate the system documentation
    $(mkdocs) build

.PHONY: rtfm
rtfm:                    # Locally read the library documentation
    $(mkdocs) serve

.PHONY: publishdocs
publishdocs: docs        # Set up the docs for publishing
    $(run) ghp-import --push site

The rtfm target is useful for locally-serving the documentation so I can live preview as I write things and update the code. The publishdocs target is used to create and push a gh-pages branch for the repository, resulting in the documentation being hosted by GitHub.

A wee problem

NOTE: I've since found out there's an easier way of fixing the issue.

This is, however, where I ran into a wee problem. I noticed that the locally-hosted version of the documentation looked great, but the version hosted on GitHub Pages was... not so great. I was seeing a load of text alignment issues, and also whole bits of text just not appearing at all.

Here's an example of what I was seeing locally:

Good layout

and here's what I was seeing being served up from GitHub Pages:

Bad layout

As you can see, in the "bad" version the func label is missing from the header, and the Parameters and Returns tables look quite messy.

I spent a little bit of time digging and, looking in Safari's console, I then noticed that I was getting a 404 on a file called _mkdocstrings.css in the assets folder. Problem found!

Only... was it though? If I looked in the gh-pages local branch the file was there (and with fine permissions). If I looked in the remote branch, it was there too. Thinking it could be some odd browser problem I even tried to grab the file back from the command line and it came back 404 as well.

Testing from the command line

At this point it was getting kind of late so I decided I must have screwed up somehow but I should leave it for the evening and head to bed. Before doing so though I decided to drop a question into the mkdocstrings discussions to see if anyone could see where I'd messed up.

As it turns out, it looked like I hadn't messed up and the reply from the always super-helpful Timothée was, in effect, "yeah, that should work fine". At least I wasn't the only one confused.

Fast forward to this morning and, with breakfast and coffee inside me, I decided to try and methodically get to the bottom of it. I wrote up the current state of understanding and looked at what might be the common cause. The thing that stood out to me was that this was a file that started with an underscore, so I did a quick search for "github pages underscore" and right away landed on this result.

Bingo!

That had to be it!

A little bit of testing later and sure enough, the documentation hosted on GitHub Pages looked exactly like the locally-hosted version.

So, TIL: by default sites hosted by GitHub Pages will pretend that any asset that starts with an underscore doesn't exist, unless you have a .nojekyll in the root of the repository, on the gh-pages branch (or whatever branch you decide to serve from).

To make this all work I added .nojekyll to docs/source and added this to mkdocs.yml:

exclude_docs: |
  !.nojekyll

All done!

And now I've worked out a simple workflow for using mkdocs/mkdocstrings for my own Python projects, in conjunction with GitHub Pages, I guess I'll start to sprinkle it over other projects too.


Hike

Posted on 2025-02-14 09:22 +0000 in Coding • Tagged with Python, terminal, textual • 1 min read

Hike

The run of writing new terminal-based tools that I want still keeps going. First there was Braindrop, then there was Peplum, and now, released today, there's Hike.

Hike is yet another terminal-based Markdown browser. While it's far from the first, and unlikely to be the last, it's mine and it looks and works exactly how I need. Perhaps it'll be your sort of thing too?

Hike viewing its README

This initial release has a bunch of handy features, including things like:

  • A command line where file names, URLs and commands can be entered.
  • A persistent history for the command line.
  • A local file browser.
  • A simple bookmarking system.
  • A browsing history.
  • Commands for quickly loading and viewing files held on GitHub, GitLab, Codeberg and Bitbucket.

As there's a lot to discover in the application, I've tried to make the help screen as comprehensive as possible:

Hike help

and there's also the command palette to help with discovering commands and the keys that are associated with them:

The command palette in action

Once again, themes are supported so no matter your taste you should find something that's easy on your eyes:

Dark Light Tokyo Night Solarized Light

Hike is licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

$ pipx install hike

It can also be installed with Homebrew by tapping davep/homebrew and then installing hike:

$ brew tap davep/homebrew
$ brew install hike

Expect to see more updates in the near future; this is very much an ongoing tinker project.


Peplum

Posted on 2025-01-25 14:21 +0000 in Coding • Tagged with Python, terminal, textual • 3 min read

Peplum

I seem to be back in the swing of writing handy (for me) little terminal-based applications again. Having not long since released Braindrop (which I'm still working on and tinkering with; it'll get more features in the near future, for sure), I had an idea for another tool I'd like to have: something for looking through, searching, and filtering Python PEPs.

As with anyone who is interested in what's happening with Python itself, I subscribe to the RSS feed of the latest Python PEPs, but I also wanted something that would let me look back at older ones in a way that worked "just so" ("just so" being "what feels right for me", of course). Having finished the main work on Braindrop I realised that the general layout of its UI would work here, as would the filtering and searching approach I used.

From this idea Peplum was born!

Peplum in action

In this first release I've simply concentrated on all things to do with grabbing the list of PEPs and presenting them in a useful way; adding various forms of filtering them; adding the ability to search the metadata; that sort of thing. I aim to keep developing this out over the next few weeks and months, adding things like the ability to make notes, to locally view the text of a PEP, perhaps even to mark PEPs as unread and read, etc.

As I mentioned earlier, much of the design was driven by what I did with Braindrop, so once again I've tried my very best to make it keyboard-friendly and as much as possible keyboard-first. This sometimes means having to work against how Textual works, but generally that isn't too tricky to do. I'm once again making heavy use of the command palette and also ensuring that all commands that have corresponding keyboard bindings are documented in the help screen.

Peplum Help

There's enough common code between Peplum and Braindrop, when it comes to this aspect of building a Textual application, that I'm minded to spin it out into a little library of its own. I'm going to sit on this code for a wee while and see how it develops, but I can see me taking this approach with future applications and doing this will stop the need to copy and paste.

It might also be useful to others building with Textual.

Also as with Braindrop, themes are a thing, and the theme setting is sticky so you can set it the once and stick with that you like. Here's some examples:

Nord Catppuccin Latte Tokyo Night

That's a small selection of the themes, with more to explore.

While working on this project I managed to find a couple more bugs in Textual, including a fun way to get transparent backgrounds to get out of sync and also a way to get an easy crash out of OptionList if it's set to width: auto.

What was even more fun was I sort of discovered a bug in the Python PEP API. Thanks to Hugo noticing my "huh, weird" post on Fosstodon, there's now an issue for it as well as a PR in the works. In retrospect I should have raised an issue myself; instead I fell into that "they obviously know what they're doing so it must be like this for a reason" trap.

Lesson relearned: it's always better to ask and get an answer, than to assume a thing is how it is for a reason you don't know; which I guess is a version of Linus' law really.

So that's v0.1.0 out in the wild. I'm pleased with how it's turned out and there's more to come. It's licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

$ pipx install peplum

It can also be installed with Homebrew by tapping davep/homebrew and then installing peplum:

$ brew tap davep/homebrew
$ brew install peplum

I'm going to be making good use of this and working on it more; I hope it's useful to someone else. :-)


Braindrop

Posted on 2025-01-03 16:24 +0000 in Coding • Tagged with Python, terminal, textual • 3 min read

Braindrop

A touch over a year ago I did the initial work on an application called Tinboard, a terminal-based client for the Pinboard bookmarking service. I had a lot of fun building it and it was an application that I used on a near-daily basis. However, around August last year I realised it was time for me to move on from Pinboard and try something new; based on various recommendations I settled on Raindrop.

As mentioned in the other blog post, Raindrop offered more or less everything I had with Pinboard and so the move was fairly straightforward. The one thing that was missing though was an application similar to Tinboard.

So, late on last year, with my winter break approaching, I decided to start from scratch and build a "Tinboard for Raindrop", which I'm calling Braindrop.

This was going to be a bit of an adventure too. Since being laid off from Textualize earlier in 2024 I'd not been following its development quite as closely as I used to, and had also run into some issues and bugs with it since that time; moreover, as well as various bugs appearing, some breaking changes had also been made. As such this was going to be a process where I'd wrap my head around what's happened with the framework over the prior six months or so.

Given all this, over the past couple of weeks I've been spending a few hours a day doing some for-pleasure coding and v0.1.0 of Braindrop is the result.

Main display

As much as possible I've tried to keep the look and feel similar to that of Tinboard, while also doing my best to avoid some of the "ah, I wish I hadn't done it this way" design decisions I'd made. As of the time of writing I'm very pleased with the result.

The edit dialog

One thing I did want to do is ensure that the application was as keyboard-friendly as possible, while also still allowing use of the mouse. Textual can sometimes get that wrong and I ran into an example of this while trying to ensure that there's good in-application help. Somewhat recently Textual added a built-in help system which, sadly, can't easily be used by and navigated by someone using the keyboard. So instead I've recreated the help system I built into Tinboard, while adopting the documentation standard that Textual had settled on (which, coincidentally, was kind of similar to what I did in Tinboard to start with).

The help dialog

As with Tinboard, I've also made sure to make full use of the command palette, with every action that makes sense having a keyboard hotkey as well as a command in the palette. I also took things a little further and made sure that the hotkeys are shown in the command palette for easier discovery.

The command palette

I've also made sure that Textual's new theme system is available for easy use; so out goes dark/light mode toggling and in comes a collection of different themes. Here's a wee selection as an example:

Example theme 1 Example theme 2 Example theme 3 Example theme 4

That's a small selection of the themes, with more to explore.

There's a few more things I want to do before I consider the application v1.0-ready, but it's already in use by me and working well. As I decide what else I want to add to it I'm building up a list of TODO items.

Given that my day job these days is quite varied, isn't quite so coding-intensive, and isn't always related to all things Python, it's actually been fun to sit down and hack up a pure Python application from scratch again. It's also helped me discover a couple or so fresh bugs in Textual (which I've reported, of course) and given me the opportunity to PR some trivial fixes as I've noticed typos and stuff as I go.

Anyway; that's v0.1.0 out in the wild. I'm pleased with how it's turned out and there's more to come. It's licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

$ pipx install braindrop

It can also be installed with Homebrew by tapping davep/homebrew and then installing braindrop:

$ brew tap davep/homebrew
$ brew install braindrop

I hope this is useful to someone else. :-)


Markdown all the things

Posted on 2024-11-04 21:00 +0000 in Coding • Tagged with Python, Twitter, Journey, Evernote • 4 min read

Recently I've been on a bit of a "turn stuff into Markdown files and slap them in an Obsidian Vault" trip. This kicked off a couple of months back when I made a decision unrelated to coding.

On and off, since my teenage years, I've kept journals. Since those teenage years it's been more off than on, but a couple of times in my adult life it's been really helpful to actually write one. The last time this happened was early 2019. It was pretty vital I did that at the time and it was a really sensible and helpful decision, and an approach to the situation I was in that I'd recommend to anyone (and have done on occasion to anyone going through the same thing).

The actual motivation for starting that particular journal is long behind me, but I'd got into the habit of writing it and so, until a couple or so months back, I kept jotting something down every day. But I came to the realisation that I didn't need to and that it had become something of a chore.

I'd been using an application called Journey. It's a great app, does the job well, but was also suffering from the creep of "AI" (I've had a few apps ion my arsenal that don't need it, acquire a useless "AI" feature). This privacy-problematic change of direction, combined with the realisation that I didn't need to write about my day, every day, any more, made me decide it was time to stop and cancel the subscription.

Thankfully Journey has a pretty comprehensive export option so I used it and didn't think too much more about it for a while.

Meanwhile I also had a subscription to Evernote that I didn't really use any more. Within it I had held a handful of years of journal entries from a decade or so ago, along with other "remember this for some point in the future" stuff. For the longest time I was on some really cheap tier that didn't exist any more, one that was low enough that I didn't really notice the cost go out each month so I kept putting off exporting things and closing it all down until "next month".

Then I got an email from them to say they were forcing me onto some new tier that was more expensive. So that was the final straw there. I made an export of what I had in Evernote and closed that account down too.

A wee while went past and then I got to thinking that it might be interesting to try and combine both these sources into one archived journal. I had stuff from around 2010 to 2015, and I also had stuff from 2019 until 2024; the former in the Evernote archive and the latter in the Journey archive. Surely I could write a couple of tools to turn that data into one consolidated Obsidian Vault?

Over the course of a couple of weekends journey2md and evernote2md were born. While both of those tools work differently, they're both designed to populate the same Obsidian Vault. Once I was happy with this I did the mass conversion and I was happy with the result.

Now I have years of journal entries, all converted to Markdown files and made available for reading via an application that lets me rummage through history using dates and tags and all sorts of other searching.

So I was happy with that and didn't give it much more though.

Then last week I got to thinking...

Twitter has turned into the worst place possible and I can't for the life of me think why any right-thinking person who has an ounce of humanity or has anything approaching a humanistic outlook on life would remain an active user. Honestly I stuck it out longer than was sensible, but in June 2023 I finally quit for good.

Back when the new owner was confirmed I, like a lot of people, extracted my archive. It's since been sat in storage doing nothing, yet there's a lot of data in there that could be interesting to work with, or just to go back and look through. So last week's thought was "why don't I also turn this into an Obsidian Vault?".

So I did...

The graph of my Twitter Obsidian Vault

The tool I built to do this is bird2glass. As you'll see in the README it makes a few assumptions about the state of Twitter archive dumps and also what a user wants from this. Personally I'm pleased with the result.

The main aim of the tool is to break the tweets down into a hierarchy of year, month and day...

Viewing a tweet

...and also to connect them with any account that was being replied to or mentioned in some way...

Viewing a user

This user view is handy when viewing backlinks, as it gives you a list of all the tweets that mention that user (and, of course, if you're into Obsidian's graph it will make for some interesting connections within there).

I sense there's more I can do with this, and I imagine I will continue to tinker with it. Meanwhile though, if that sounds like something you'd benefit from do feel free to grab it and play with it and hack on it. Keep in mind the notes and assumptions that are in the README, and really be prepared for a lot of files to be created if you did a lot of tweeting like I did (I do think that over 50,000 individual files for an Obsidian Vault is a bit silly, if I'm honest).

Meanwhile... I might need to look at other applications and think about how I can turn the data into useful Markdown collections!


Paindrop v1.0.0

Posted on 2024-08-18 10:57 +0100 in Coding • Tagged with Python, pinboard, raindrop • 4 min read

I was quite late discovering Pinboard; by the looks of things I created my account and paid my first subscription for it in early 2019. Since then I've been a pretty avid user and found it really useful. I've even written a couple of clients for it (one for Emacs and one for the terminal).

During that time it's had its fair share of hiccups and outages, but on the whole I've found it a stable and useful service.

The service does have its detractors, and concerns over its long-term stability and how well it's maintained are fairly common. I half paid attention to these, and had started to think about where I might go if there was an issue.

While maintaining and syncing bookmarks isn't exactly a difficult or unsolved problem, and while it's also true that it could be fun to roll my own solution, there are a couple of things I need that would make building my own approach a bit of a chore.

Things important to me are:

  • An extension for any random browser I might find myself using
  • A good mobile client for at least iOS and iPadOS
  • A good API so I can write my own tools if I need to
  • A clean and focused backend website

I kept these things in mind and kept an eye out but I'd never really felt the need to actively start looking around.

Then I stumbled on this after posting about another Pinboard outage.

That... yeah, that was the final push I needed to start to think seriously about where to move and how.

A couple of people suggested Raindrop, and from what I could tell it was coming up as a pretty popular service that some Pinboard users were migrating to. I had a look and it wasn't quite what I was after; but close.

You see, there's two things I really like about Pinboard that Raindop didn't seem to cover:

  • Simple support for "this shit is unread". I see things, I share to whatever Pinboard app I have on my phone or tablet, etc, and then I review some time later (normally in Tinboard).
  • Support for Private and Public pins. I've liked having a feed of bookmarks I can let people see and Raindrop doesn't have this.

I looked around at some blogs that talked about Pinboard vs Raindrop and didn't see any that really dived into this particular aspect of migrating; I also asked a couple of folk who'd made the move about this and they didn't really have any insight (mainly because they didn't care about those particular uses).

One thing I did notice though was that Raindrop does support making individual collections public. So, if I was willing to sacrifice any other uses for collections (a bookmark in Raindrop can only be in one collection), I could simply have a Public and a Private collection and import pins into the appropriate one. Also, unread pins could be left out of the collections and I could use that to signify unread status.

This seemed fine as I'm heavy on the tags anyway.

Now... Raindrop has a pretty comprehensive import facility built in. I gave it a try with Pinboard's backup file and it worked really well. That is... really well except it just threw away the public/private/unread aspect of the pins. There was only one thing for it then: I had to write my own importer!

Which brings me to Paindrop. It's a quick hack but it does the job, and it does the import just how I wanted. The result of the first test was pretty much spot on (in this image I'm comparing what Raindrop says vs what Tinboard says I have in Pinboard):

Comparing Raindrop and Tinboard contents

Usage is pretty straightforward. You create Public and Private collections in Raindrop, you create an app in Raindrop and get the access token, you grab your Pinboard access token and then:

$ paindrop example:xxxxxxxxxxxxxxxxxxxx xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

where the first parameter is the Pinboard access token and the second the Raindrop access token.

If all goes well, after a few moments, the importer should finish and you should find that all of your pins have migrated to Raindrop, all public pins are in the Public collection and all private pins are in the Private collection. Any pins that were marked as unread will be Unsorted.

Note that if you used different names for your public and private collections you can pass those names to paindrop with the --public and --private switches.

If you're looking to move your bookmarking history out of Pinboard and want to keep the same sort of structure I had I hope Paindrop will be useful to you too.

Paindrop can be installed with pip or (ideally) pipx from PyPi. It can also be installed with Homebrew by tapping davep/homebrew and then installing paindrop:

$ brew tap davep/homebrew
$ brew install paindrop

The source is available on GitHub.

PS: As for the name... originally it was pin2rain but then Darren Burns pointed out the obvious and it had to happen.