Posts in category "Coding"

I've just released v2.0.0 of Rogallo. This release has quite a lot of changes, one or two of them "breaking changes", hence the bump to v2.0.0. I will add though that, when I say "breaking changes", nothing should actually break, it's just some features have changed in an incompatible way and some application commands have been removed or renamed (and so some default keyboard bindings have changed). The worst that might happen is a keyboard binding you set up no longer works.

With this in mind, in the spirit of semver, the major version is bumped.

Now for what's new in this release.

Client certificate management¶

While Rogallo supported the creation and use of client certificates for use with Gemini capsules, it never made it that easy to work with them. Creating and using them was pretty frictionless, but if you were someone who wanted to bring your own certificate from another application, you were on your own. Actually doing that was easy enough, if you were comfortable with diving into the certificate directory and hand-editing the JSON file to create the association. What was lacking though was a good interface to manage all of this in the application.

This version solves that problem. Rogallo now contains a client certificate manager that lets you:

• Create new client certificates
• Add and remove associations for client certificates
• Delete client certificates
• Import client certificates
• Export client certificates

Also, when you encounter a capsule that demands a certificate and you don't have one associated, you can pick an existing one from your certificate library; before the only option you had was to create a new one.

Reworked side panel¶

Initially, when I first started work on Rogallo, I had a simple history list that could be popped in/out on the side of the display. Then, when I added bookmarks, this became a sort of toggle, with either the history showing, the bookmarks showing, or both hidden. The command to show the history would hide the bookmarks; the command to show the bookmarks would hide the history; both would toggle themselves to hide. This was... kind of messy, but it worked.

When I started work on the certificate manager it made sense that it also occupied the same space. However, the code for managing this, and the resulting user experience, started to get pretty messy. I wasn't at all happy with it so ended up going on a bit of a side-quest to clean this up.

The result is that Rogallo now has a tabbed side panel (similar to the one in Hike). This contains the bookmark manager, history manager and client certificate manager.

This is where one of the main breaking changes comes in: the ToggleHistoryManager and ToggleBookmarksManager commands have been removed from Rogallo and their default keyboard bindings have been freed up. Also, to clean up the naming of things, I've renamed JumpToSidebar to JumpToSidePanel.

I feel this side panel approach is far cleaner and far easier to work with.

It's also worth noting that it's moved over to the left side of the display. However, if you'd prefer it on the right there's a configuration option for that.

Viewing client certificate information¶

As part of the work on managing client certificates, I added a dialog that lets you view details of a certificate. As well as being available in the certificate manager, it can also be used to view any certificate that might be in use when viewing the current page. The AboutClientCertificate command (bound to Shift+F7 by default) can be used to bring up the view.

For the folk who lean more on the mouse, the key icon that appears in the title bar of the viewer can be clicked on to get the same view.

Protocol guessing in the command line¶

The command line in the application now tries to be a little bit smarter when it comes to understanding input. Before now, if you were to type in example.com, Rogallo would assume that you were attempting to visit a Gemini capsule on that host and process the input as if it were gemini://example.com/. This release extends this a little.

• If the input doesn't appear to be anything else and it includes a @ it will be turned into a finger URI. So user@example.com becomes finger://example.com/user and @example.com becomes finger://example.com/.
• If the first part of the host has a recognisable protocol name, it will be assumed that that protocol should be used. In other words:
  • gopher.example.com -> gopher://gopher.example.com/1
  • spartan.example.com -> spartan://spartan.example.com/
  • nex.example.com -> nex://nex.example.com/

This is probably of very minor use, but it was easy enough to add and seemed like a useful little change.

Emoji removal overhaul¶

I've overhauled the code that does emoji removal if you toggle removal on. More work is done to identify "real" emoji. This should mean that all the usual colourful image-like candidates are removed, but now things like braille and box drawing characters, etc, won't be removed.

Also, when removing an emoji, if there is a space following it, that space will also be removed. This solves the cosmetic problem of "👉 This" being turned into " This" when it would make more sense that it is turned into "This".

It's worth keeping in mind here that this isn't intended to be a 100%-correct solution. It's aimed at being a good-enough solution to clean up most of the emoji clutter if you don't like that sort of thing.

Overall performance¶

My general approach to developing anything is one of "make it right then make it fast". I've never been a fan of early optimisation, and when I've watched people obsess over such things far too early in a codebase's lifecycle I've generally watched them get into a mess. I like to try and avoid that mess.

So this was my approach with Rogallo. When it came to building the widget that displayed the content of a page, I spent my time trying to ensure it was correct, not that it was fast. Quite quickly, however, I found that once a page had quite a few links on it -- especially when you got to many 100s of links -- performance was terrible. I'd had an issue sat in TODO for this from the start, but hadn't gone back to address it.

Then the other day someone else noted this issue so I decided it was time to go back and work on it. As of this release of Rogallo you should find that a page with 1,000 links on it performs about as well as a page with a handful of links.

In doing this work I've also made a lot of optimisations to how a page is initially built; so not only should a page be more responsive as you navigate it, it should be noticeably quicker to appear in the first place.

Conclusion¶

There's a lot in this release and I'm really delighted with how it's turned out. I wasn't intending to make quite so many large changes, but each bit of work naturally caused the next bit of work and... well, here we are.

With this work out of the way I can start to look at the one remaining protocol that's currently on the TODO list, and also look at making some other quality-of-life improvements.

I've just made a small but important bug fix to Rogallo. Currently, I'm working on a more comprehensive approach to managing client-side certificates and, in doing so, I noted a bug with how such certificates are currently created by Rogallo.

Simply put: by default, Rogallo should create certificates that effectively never expire (the actual expiry time is the end of 9999-12-31). Instead, if no actual expiry time was given, the certificates were being created to expire a year after creation.

This does mean that, if you've created a client certificate with Rogallo and didn't specify your own expiry date, it currently has a far more limited lifespan than you were probably expecting. As such, you might want to review them and consider making fresh ones and setting them up well before they expire (expiry probably being some time in July next year, given that Rogallo itself has only been fully usable since around July this year).

Apologies if you've used this facility and it accordingly results in a little extra work. Thankfully it came to light sooner rather than later, and it has come to light while I'm giving client certificate management a big overhaul.

I've released v1.12.0 of Rogallo. This version contains a number of internal changes, as well as some user-facing improvements.

Improved navigation history¶

For a wee while now I've been wanting to tweak the way the navigation history works. Up until this release, this was simply a backward/forward affair with no attempt to restore the actual context of a page you've gone back to. This could best be felt if you're a fan of AstroBotany. If you were to visit the flowering plants in your community garden, with a view to visiting each one and picking petals, the natural thing to do would be to visit a specific plant, pick a petal, then go Backward twice to get back to the list of flowering plants. The problem at that point is that the link for the just-visited plant would no longer be focused. This made it just a bit more faff to go to the next plant on the list.

From this release, when going backward and forward through navigation history, Rogallo attempts to ensure that the link you had followed on that page is focused again.

Internally, this caused a fairly big overhaul of how history navigation information is recorded. I've done a fair bit of testing and all seems good; but do let me know if you experience any oddness.

Showing empty content¶

This is a small fix. If you were to visit a URI that resulted in an empty document being received, the viewer would disappear. The viewer is supposed to disappear if you are visiting nowhere, but it shouldn't disappear if you're visiting somewhere that has nothing to show. This is now fixed.

Parent/root navigation¶

Rogallo has a couple of commands that make it easier to navigate to the parent directory of a visited location, and also to the root. However, I'd only implemented this for Gemini and Spartan URIs and sort of forgotten to finish this off. With this release Gopher and Nex get to join in on the parent/root navigation party.

Cancelling requests¶

This one came up because, as of the time of writing, the server where my wee capsule lives is dead¹. This meant that if I checked if it was there, and got impatient and went to navigate somewhere else, the somewhere else would load and then, after the defined timeout, the error about the previous request would pop up.

Rogallo now ensures that, if you have an in-flight request running, and then navigate elsewhere, that pre-existing request is cancelled.

Custom prompts¶

The in-application command line of Rogallo has two styles of prompt. The normal input prompt (shown as >), and the busy prompt (shown as a series of braille characters that animate as dots snaking around). I realised that some folk might enjoy setting their own prompts. While I think the defaults are nice and clean, if you wanted something more colourful and on-theme (in a Gemini sense anyway, which tends to be space-oriented), you could update your configuration file with:

    "command_line_prompt": "🚀",
    "busy_indicator_cells": "🌑🌒🌓🌔🌕🌖🌗🌘"

and have this:

I'm currently running with this for my own installation of Rogallo and it's rather growing on me.

A quick little update to Rogallo, bumping to v1.11.0. This release is mainly in support of people who like to use the mouse as much as, if not more than, the keyboard; and also in support of people who might wish to reclaim a couple of lines of display for actual content.

The main change in this release is that I've swapped out the header line that Rogallo used to have (which used the stock Textual Header widget) and replaced it with a mouse-friendly toolbar.

The thinking behind this was twofold: first, the current header wasn't really very useful, showing just the name of the application and the version number. I would hope, really, that most people know what application it is they're running, so having the name there wasn't super useful. Secondly, while Rogallo aims to be keyboard-friendly -- ideally keyboard-first -- it also aims to be mouse-friendly where possible. The problem with the mouse-friendly approach is that a lot of application commands were, for the mouse user, locked behind the command palette. While not impossible to navigate and use, it wasn't the smoothest experience.

So I've swapped that header out for a configurable toolbar of command buttons. The content of the toolbar is set using the toolbar_contents setting in the configuration file. The default set looks like this:

[
    ["GoHome", "\u2302"],
    ["Reload", "\u21bb"],
    ["Backward", "\u25c0\u25c0"],
    ["Forward", "\u25b6\u25b6"],
    ["GoToParent", "\u2191"],
    ["GoToRoot", "\u21c8"],
    ["SearchHistory", "\u25f7"],
    ["SearchBookmarks", "\u2605"],
    ["ToggleView", "\u21cb"]
]

The value in the first position for each button is the name of a bindable command. The second value is the text to place on the button itself. If you simply want to use the command's name, rather than a pair of values, you can just use the command name. So suppose you wanted to use Reload, Backward and Forward as-is, you could set things like this:

[
    ["GoHome", "\u2302"],
    "Reload",
    "Backward",
    "Forward",
    ["GoToParent", "\u2191"],
    ["GoToRoot", "\u21c8"],
    ["SearchHistory", "\u25f7"],
    ["SearchBookmarks", "\u2605"],
    ["ToggleView", "\u21cb"]
]

In addition to this setting, I've also added some other toolbar-related settings. There is:

"toolbar_visible": true

which controls if the toolbar is visible at all. So if you're a keyboard-only kind of user and you want that extra line back, just set that to false.

There is also:

"toolbar_tooltips": true,

which controls the tooltips that will appear when you hover over a button in the toolbar. These are there to explain what each button does, and to also show you what keyboard binding is associated with each one. If the tooltips get irritating, set this to false so they don't appear.

Lastly, there is:

"toolbar_can_get_focus": false

As I've said: the toolbar is added in support of people who use the mouse a fair bit. This means that, by default, it can't be interacted with using the keyboard. This makes sense in that each of the commands will have a keyboard binding, so it makes more sense to simply use that. However, if you really must use the toolbar with the keyboard, you can set this setting to true to enable it. When set this way, you can navigate into the toolbar and switch focus to each of those buttons using the normal Tab and Shift+Tab navigation keys.

One other little configuration feature can also be used to reclaim an extra line in the display. If you have no need for the footer of the application (where some important keys are shown), you can turn it off with:

"footer_visible": true

Set it to false and you get one more line of content.

Rogallo v1.10.0 has been released. This is a pretty small release, with a bug fix to navigation, and an improvement to how navigable Markdown documents are if viewed in the application.

In Rogallo v1.8.0 I made some changes to how Markdown documents are shown, if they're encountered and detected in Geminispace (or any of the other protocol spaces that might let you know the content type of a document). Whereas originally the document was shown in its raw form (marked up with syntax highlighting), v1.8.0 moved to using a fully-rendered form thanks to the internal Markdown widget.

But, as I've mentioned elsewhere in this blog, I wasn't 100% happy with this. The main problem is that the Markdown widget supplied by Textual is pretty unfriendly to keyboard navigation, and I want as much of Rogallo to be navigable with the keyboard as is possible.

To solve this I've created md2gemtext and started using it here in Rogallo. This means that, by default, from now on, any time a Markdown document is received by Rogallo, the content is converted into Gemtext before being displayed. The big advantage here is that links are far easier to see and follow and the document as a whole looks more in keeping with most other pages you'll visit.

I'm sensitive to the fact that some folk might prefer the previous method of viewing Markdown, so this feature is configurable. If you'd prefer the widget-based rendering instead, set this value:

"convert_markdown_to_gemtext": true

to false.

The other change in this release is a fix to how navigation history is saved and restored. For a couple or so weeks now, on occasion, I was finding that when I ran up Rogallo again, navigated from the restored page, and then used Backward, I would end up somewhere else. It was intermittent and, for a while, hard to pin down. Mostly I'd notice this while rapidly closing and opening Rogallo while working on some other feature. Eventually I figured out the sequence of events causing the problem and it was an easy enough fix.

Another update to Rogallo, another new protocol! The main change in v1.9.0 is the addition of support for the Nex protocol.

As with any other supported protocol, entering a nex:// URI into the command line will cause Rogallo to load up the response in the viewer and render it. Nex is kind of fun in that it's almost a plain text system, by default, but the expectation is that any text/plain document that has => at the start of a line will have those lines be treated just like Gemtext links. So, in Rogallo, when you visit a page, and that page is text/plain, the result will be rendered like it's very simplistic Gemtext that only supports links.

Also, as per the Nex specification, the MIME type of a document is worked out from the extension of the file it points at; so if you happen to get a Gemtext or Markdown file back from a nex:// URI, this will be suitably rendered.

Another addition is a SaveSource command (bound to Ctrl+s by default). As you can probably work out from the name, this will save the source of the document you're currently viewing.

A small housekeeping addition in this release is the cleaning of the content cache. When you start up Rogallo, in the background, any cache entries that have exceeded their time-to-live (something you can set in the configuration file) will be removed. If this results in any cache directories being left empty, they will also be removed. This should result in a nice clean cache directory that's easier to navigate and manage, if you want to go diving into it.

I've also added a small experimental development tool, mostly for myself but it might be something someone else wants to play with. I realised a few days ago that it could be fun to include screenshots of Rogallo in action in my Gemini capsule. The thing is: Gemtext doesn't allow for inline images. What it does allow for, and many clients support, is ANSI escape sequences. Meanwhile, Rogallo is a terminal-based application; its whole method of display is ANSI escape sequences. As such, it should be trivial to capture the sequences for any given screen and just include them in a Gemtext page.

I tried it out and it works a treat! So, if you happen to have ROGALLO_SCREENSHOTS in your environment and it's set to any non-empty value, pressing Ctrl+Shift+F12 writes the current screen to ~/rogallo-screenshot.ansi.

Yes, it's all horribly hard-coded: the key combination is hard-coded (and probably only good for more modern terminal emulators), and the name of the file is hard-coded. As I said: it's a development tool that's there for my own use. I might make it a little more generic in the future.

One final tweak to this release is a fix to the ToggleView command, which stopped letting you view the source of a Gopher map. That ability is now back.

Currently, I'm working on adding support for the Nex Protocol to Rogallo. Having Port1900 as the client library, and having done the bulk of the work in Rogallo, I needed things in place to be able to add some nex://-related documentation too -- which is where SmolServe comes in.

So I've dropped v1.1.0 on PyPI, which adds support for running up a simple just-good-enough Nex server (although, to be fair, Nex is so simple this is probably a fully capable one just by the very nature of how trivial the protocol is).

Now to get back to polishing that Rogallo PR...

I've released v1.8.0 of Rogallo. This release has a bit of an accidental emphasis on control of how things are rendered in the main viewer.

Markdown¶

While Rogallo is, of course, primarily aimed at loading and rendering Gemtext documents, I have written it so that it'll try and display anything whose content type matches text/*. In most cases this will result in something being shown as plain text and, as of v1.6.0, it will also attempt to add some syntax highlighting where appropriate. So, if you happened to come across a Markdown file in Geminispace, and the server told Rogallo that it was a Markdown file, it would look something like this:

Simply put: raw Markdown but syntax-highlighted.

The thing is, Rogallo contains everything needed to actually render the Markdown; it's carrying that code around due to it being built on top of the Textual framework, so it seemed a shame to not use it. So as of this version, if a Markdown file is encountered, by default it'll be shown as a full Markdown widget.

HTML¶

Having added the above, I got to thinking that it might be interesting to consider other text/* types that I could render in a more Rogallo-friendly way. The next that came to mind was text/html. While it's not going to be that common to encounter HTML on the end of a gemini:// connection, it's not impossible. Initially I did consider going via a route that had me turn the HTML into Markdown and then use the Markdown widget to show the result, but in the end I decided to turn it into Gemtext instead; in part because it's a fun challenge, and in part because it feels more in keeping with the Geminispace aesthetic¹.

So, whereas before, if you landed on some HTML in Geminispace, you'd see this:

From now on you'll see something more like this:

This HTML to Gemtext support is still in its infancy so I imagine there's going to be a lot of fun corner cases that don't quite work out, but for me the utility makes it worth working on.

Extended ToggleView¶

Until now the ToggleView command (bound to F4 by default) simply toggled Gemini-based Gemtext and Gopher maps between their rendered and raw views (acting as a sort of "view source" command). With the addition of richer rendering of Markdown and HTML it will also toggle those content types. The point here being, if you really need to look at the raw content of a Markdown or HTML document found in Geminispace, it's still there for you.

The HandOffToOperatingSystem command¶

A key feature of Rogallo is the automatic handing off of URIs and MIME types that it can't directly handle. On occasion, though, I find myself wanting to do the same hand-off with a document I'm successfully viewing in Rogallo. This was easy enough to do with the CopyLocationToClipboard command (bound to Ctrl+Shift+c by default), pasting the URI into my normal web browser and going from there. But it felt like this should be a command in its own right.

So now there's HandOffToOperatingSystem (bound to Ctrl+Shift+o by default), which will ask your environment to open the current URI.

Hiding some pre-formatted text¶

Yesterday, I saw a post on Station which was talking about the site logo. The main thrust of it being, while the logo is cool and all, it does use up a fair bit of vertical space that you have to get past to get to the content. I can see this being an a11y issue in some situations, if nothing else.

So I got to thinking: in the case of Station, the author of the site has been kind enough to give the pre-formatted text block some alt-text², so that means it's possible to have a targeted filter for such well-designed sites. Given this, I've added a method of filtering out pre-formatted blocks, keyed on a URI and alt-text. So now, with the addition of this to my configuration:

"hide_preformatted": [
    [
        "gemini://station.martinrue.com/",
        "Station logo"
    ]
]

I can get straight to the content when landing at the Station:

Gopher cache support¶

More as an accident of how I was working on things than a conscious choice, the only protocols in Rogallo that also used the content cache were Gemini and Spartan. With this release of Rogallo I've extended it to Gopher responses too. I've made a point of not using the cache for any response that is going to be the result of a search, as that feels like something that's often going to have more dynamic content.

Small Gopher fix¶

Talking of Gopher: there's also a small fix to Gopher responses in this release. When looking for responses that indicated an error from Gopher servers, Rogallo was a little too keen in how it worked, which could result in a plain text response, that just happened to have a 3 in the first column of any line, being seen as a Gopher map that indicated an error.

This is now fixed.

I've just made a small release of Rogallo, bumping the version to v1.7.0. Most of the changes in this version aren't very visible, but quite a few things have changed under the hood.

One change that might show up on occasion is the addition of an encoding fallback if you encounter an ISO-8859-1 file out in the wild (oh how that takes me back). Before, Rogallo would pop up an error and refuse to show the file; now it will show it just fine.

I've also made some improvements to how Rogallo navigates Gemtext files being browsed locally in the filesystem. Before now, when viewing local files, if a link was for another local file, it would show an external link icon rather than a Gemini link icon. This is now fixed. The other change is that, if you're browsing local files that have links to a directory rather than a file -- thereby implying that an index.gmi should be looked for -- Rogallo will take a peek at the directory, see if there is an index file available, and take this into account.

This should make it a smoother experience if you're using Rogallo to locally preview some Gemtext files before you upload them to a capsule.

The final noticeable change is the addition of a ViewChangeLog command (AKA !view_change_log in the internal command line). Bound to Ctrl+Shift+l by default, this navigates you to the Gemtext version of the Rogallo ChangeLog.

In addition to this, I've made a lot of changes to some of the internals. The class that handles the main screen of Rogallo was getting pretty large, with a lot of detail of how to handle different protocols sitting in the code. While not causing any runtime issues, it made the code increasingly bloated and untidy (and I really hate large Python files). So I've moved the handling of each of the protocols into their own files. This also has the benefit of making it a little easier and a little more maintainable if/when I add another protocol or two. It should also make the code more readable for anyone else looking over it.

As Rogallo got close to being "stable", I did a fair bit of work on its documentation. Something that I wanted to include in the site was a good collection of up-to-date screenshots. These are all created on the fly. To do this, of course, requires something that looks like a Gemini server.

Initially, this was simple: Rogallo only supported Gemini capsules, so I could illustrate most things just using Gemtext files in the local filesystem (with an admonition in the documentation to point out that Rogallo was for more than viewing local files). This, of course, wasn't going to scale when I added Finger support, and neither was it going to work well for Gopher support.

The solution seemed obvious: use a lightweight local server for these protocols. With this need in place SmolServe was born. As mentioned when I released Rogallo v1.5.0, this isn't a project to build a comprehensive smolweb server. The aim is to build myself a minimal just-good-enough server that helps me with local testing and documentation.

As it stands, SmolServe supports Gemini, Gopher, Finger and Spartan. Or, rather, it supports a just-enough-to-get-by version of each of those protocols. None are implemented in a way that would serve as a "production" server; they're implemented to allow me to generate screenshots for the Rogallo documentation, involving any of the supported protocols, without the need to rely on services I don't control and which aren't local.

The big benefit of SmolServe is the exec support. With this, you can run up the server and then have it run another command. Once that command finishes its work, SmolServe will close down too. This means that, when it comes to producing the Rogallo documentation, I can just have the server running when I need it. Pulling some snippets from the Rogallo Makefile:

run      := uv run
smol     := $(run) smolserve --config $(docs)server/smolserve.toml
smolexec := $(smol) exec --
mkdocs   := $(smolexec) mkdocs

##############################################################################
# Documentation.
.PHONY: docs
docs:
    $(mkdocs) build

.PHONY: rtfm
rtfm:
    $(mkdocs) serve --livereload

.PHONY: publishdocs
publishdocs: clean-docs
    $(mkdocs) gh-deploy

The idea is that, when I build the documentation, I actually run smolserve, which in turn runs mkdocs, which then produces the documentation while the local server is available.

I also use this sort of approach for local testing of the content of my capsule that lives over on tilde.team.

.PHONY: view
view:
    uv run smolserve --config smolserve.toml exec rogallo open gemini://localhost/

My aim now is that, if I add any other protocols to Rogallo, I'll add a just-good-enough version of them to SmolServe to help me with testing and documentation. For the moment, though, I think it's in a stable and usable state.