Posts in category "Python"

I've just made an initial release of a new library related to my ongoing project to build my own Gemini protocol browser. Initially, the code to parse the hypertext format used for Gemini sites, lived in the Rogallo codebase. But despite it being a pretty simple bit of code, I felt it could be useful for other things too. So rather than have it be buried inside a package that has a lot of other dependencies, I've decided to spin it out into its own little package.

So gemtext v0.1.0 is now available. The library provides a single parsing class, which takes raw markup as a string and turns it into a sequence of objects, each typed for the type of line found. A very simple parsing tool might look like this:

import fileinput
from gemtext import Gemtext

def parse_input() -> None:
    for gem_line in Gemtext("".join(fileinput.input())).content:
        print(f"{gem_line!r}")

If fed with the following input:

# This is a heading

## This is a sub-heading

### This is a sub-sub-heading

=> gemini://davep.gemcities.com/ Dave's test capsule

> This is a deep and meaningful quote

```
Here is some pre-formatted text.

Here's some more of that text.
```

* One
* Two
* Three

the output would be this:

Heading(content='This is a heading', level=1)
Paragraph(content='')
Heading(content='This is a sub-heading', level=2)
Paragraph(content='')
Heading(content='This is a sub-sub-heading', level=3)
Paragraph(content='')
Link(content="Dave's test capsule", uri='gemini://davep.gemcities.com/')
Paragraph(content='')
Quote(content='This is a deep and meaningful quote')
Paragraph(content='')
PreFormatted(content="Here is some pre-formatted text.\n\nHere's some more of that text.")
Paragraph(content='')
ListItem(content='One')
ListItem(content='Two')
ListItem(content='Three')

That's the extent of the library for the moment. I don't see it growing too much, given how straightforward the markup language is. Perhaps one addition I might make at some point is a method of going the other way: allow collecting together each of the individual line-oriented objects and getting a text document back, so providing an object-oriented interface for producing Gemtext documents.

For now though this is enough to support what Rogallo needs.

I've just released v0.1.0 of Wasat, my async Gemini Protocol client library for Python.

Changes in this release include:

• Support for generating and storing client certificates to help when handling 6x responses. This is still experimental.
• Updated the CLI to handle requests for input (handling 1x responses).
• Added a uri property to the Response class, to expose the target URI reached from a request.
• Added a history property to the Response class, to expose the redirect history if redirection took place.
• Added a requested_uri property to the Response class, to expose the originally requested URI.
• Updated the CLI so that, when in verbose mode, it prints all of the available redirection information.

Most of the changes here are in support of resolving an issue I found with Rogallo yesterday. With v0.1.0 available I should be able to update Rogallo with an easy fix.

So far, building this library, and the client application, is proving to be really interesting and educational. There's something fun about building a "web browser" of sorts, from the ground up. It really hits this point:

{Gemini might be of interest to you if you} Are a hobbyist programmer with a "do it yourself" attitude who enjoys building their own tools and getting real use out of them every day

from the Gemini protocol FAQ. For me, in "hobbyist programmer" mode, this is all kinds of fun.

BagOfStuff started as a very small side project when I was working on OldNews during my winter break, and then into the new year. It began as some support code in OldNews, which was living in a sort of tools section of the codebase (you know: the dreaded and much-maligned utils section of the project). Sensing that something in it could end up being of utility elsewhere, I moved it out into its own library.

Having written some history-management type of code for the Gemini protocol client I'm working on, I got to thinking that it too should move into this library; a benefit here being that I'll eventually migrate Hike's histories to this. With that done, I've decided to promote BagOfStuff to v1.0.0 and call it stable.

I don't see this library growing too much; the old days of languages either having a spartan standard library, or a third-party ecosystem that lacks comprehensive coverage, are pretty far behind us, and Python suffers from neither issue. On the other hand, there are some general-but-niche things that I'm going to want for my own projects, which probably don't deserve a library in their own right, which can live in here.

Also, sometimes, it's fun to just jam on a little problem and try to make the personally-ideal approach to solving it. BagOfStuff serves that purpose too.

I can't remember how and where I saw it, but just over a week ago I ran into Project Gemini. Somehow I've never read or seen anything about this before, which is pretty wild considering it's been on the go for around seven years now. As I read up on it I got more and more intrigued. I had the urge to do... something with it.

I think the thing I want to do, and I know I'm far from the first, is write a client for the terminal. I'm envisioning something very similar to Hike, but obviously only targeting the gemini protocol itself, and only handling and rendering gemtext.

I will, obviously, be looking to write this in Python, and of course, will be looking to use Textual, which means that it would be useful to have a Python client library for the protocol, and ideally a client library that is async. I did some searching, found client libraries, but none of them seemed to be async-first.

With this in mind, and to kick-start the project, last night I fired up Antigravity¹ and got a library up and going. Wasat is the result. For the moment this should be considered alpha-status software (hence v0.0.1). I've done some very rudimentary testing and experimenting with it and, so far, so good. It's also proving to be a good tool with which to get to know the protocol. It also gives me another project to use to experiment with an agent (this being the first project I've started from scratch using Antigravity).

Over the next few days I'm going to toy with the library more, clean up the code, look for any issues, and then I'll start on the client application. That will be hand-built; no AI. I have some ideas of fun things I want to do, especially when it comes to handling gemtext.

It'll be nice to have a new pet project that's a hand-coded project. The first significant one since OldNews.

Getting back to Wasat itself: I believe it has everything necessary to allow for writing such a client (which, to be fair, isn't much -- that's kind of the point of the protocol). It also comes with a simple CLI built in, which can be run with python -m wasat (if it's just installed as a library) or with wasat (if installed globally, along with any command scripts). The command itself is just a simple download tool for a "page" in a "capsule". For example, I can grab the content of a test capsule I've created:

$ wasat gemini://davep.gemcities.com/
# Introduction
Hey! I'm Dave. Normally you'd find me at:
=> https://www.davep.org My web site
or
=> https://blog.davep.org My blog
amongst other places. But I discovered Gemini and I'm really curious about
the idea, so here I am giving GemCities a go to get to know things a little
better.

(with thanks to GemCities for providing a neat little service).

As for where this is all going: there's no direction, really. I've found a neat new thing that I didn't know about before, the idea sits well with me, and I want to explore it more. It also gives me an excuse to do a thing I really enjoy doing: writing terminal-based TUI applications for the sake of it, and especially writing one that works just how I want.

I had quite the adventure with PyPI this morning, and I don't think it's over yet. It started out with the release of BlogMore v2.43.0. I did my usual thing of doing a test release to the test version of PyPI, and then did a production release.

As normally happens, I then went on to tag the release on GitHub, followed by writing the blog post to announce the new version. While doing this, despite the fact that it wasn't necessary given the nature of the change, I decided to update BlogMore in my blog's repository. That's when things started to look odd.

I did the usual make update but nothing new appeared. Now, it's not unheard of that I do this and no new version of BlogMore appears. Often I do it a couple more times and it's fine. So I kept trying every minute or two and still nothing. So I checked back on PyPI. Sure enough, a search showed that it had updated:

(The 16 minutes being about the time since I'd made the release), but when I clicked through it was showing the last version from a couple of days ago. Even when I looked at the release history it was saying the latest version was the previous version:

Odd.

At this point, depending on how I searched and where I went, I'd either see that my latest upload wasn't available, or I'd get a 500 error.

Clicking through to the status page showed no errors. Clicking through to the Twitter account that was suggested showed nothing at all.

Leaving aside the whole issue of having an account on Twitter these days anyway, I felt it wasn't that useful to point people at a resource that seems to have never been updated, so I did raise an issue about that.

Digging around the status page at some point, despite the fact that the main display was green all the way, I did see a rise in "PyPI CDN Edge Errors". I'm not a web guy, I'm not an infrastructure guy, so I'm not really sure what this would mean, but it sounds like it's not a good thing.

Opening the graph to look longer term, it did seem today was a spike, with another spike quite some time ago.

At this point I left it a while, not announcing the new version of BlogMore. I came back some time later and, finally, I could see 2.43.0 was showing! Also, this seemed to coincide with the above graph calming down again.

Seeing this I went to upgrade BlogMore in my blog's repo/venv and this time it all worked.

Yay!

At that point I left it alone and went about my work day. However, I don't think whatever is going on is over. Despite the fact that it was showing BlogMore as being v2.43.0 earlier today, once things were settled, I just checked again as I started to write this and:

The search index on PyPI shows it as having been updated about 8 hours ago (as I write this), but the page itself shows that the latest version is from 2 days ago. At least installing it gives me 2.43.0:

$ uv add blogmore
Using CPython 3.13.1
Creating virtual environment at: .venv
Resolved 17 packages in 325ms
Installed 16 packages in 41ms
 + blogmore==2.43.0
 + feedgen==1.0.0
 + jinja2==3.1.6
 + lxml==6.1.1
 + markdown==3.10.2
 + markupsafe==3.0.3
 + minify-html==0.18.1
 + pillow==12.2.0
 + pygments==2.20.0
 + python-dateutil==2.9.0.post0
 + python-frontmatter==1.3.0
 + pyyaml==6.0.3
 + rcssmin==1.2.2
 + rjsmin==1.2.5
 + six==1.17.0
 + watchdog==6.0.0

Also: PISpy sees 2.43.0 as the latest version too (something it wasn't seeing during the height of the issues this morning).

It's all kind of confusing though.

Guess it's time for me to read up on CDN edge errors or something...

Last week I was wrestling with some Textual code, trying to get something to lay out on the screen "just so". On the whole this isn't too tricky at all, and for those times where it might feel tricky there's some advice available on how to go about it. But in this case I was trying to do a couple of "on the edge" things and one thing I really needed to know was what particular part of the display was being "caused" by what container or widget¹.

Now, at the moment anyway, Textual doesn't have a full-blown devtools with all the bells and whistles; not like in your average web browser. It does have a devtools, but not with all the fancy DOM-diving stuff the above would have needed.

What I needed was the equivalent of print-debugging but with a point-and-ask interface. Now, I actually do often do print-debugging with Textual apps only I use notify; this time though notify wasn't going to cut it.

I needed something that would let me point at a widget and say "show me stuff about this". Something that happens when the mouse hovers over a widget. Something like... a tooltip!

So that was easy:

def on_mount(self) -> None:
    for widget in [self, *self.query("*")]:
        widget.tooltip = "\n".join(
            f"{node!r}" for node in widget.ancestors_with_self
        )

Suddenly I could hover my mouse over a bit of space on the screen and get a "traceback" of sorts for what "caused" it.

I posted this little hack to #show-and-tell on the Discord server and someone mentioned it would be handy if it also showed the CSS for the widget too. That was simple enough because every widget has a styles.css property that is the CSS for the widget, as a string.

After that I didn't think much more about it; until today.

Looking back, one thing I realised is that adding the CSS information on_mount wasn't quite good enough, as it would only show me the state of CSS when the mount happened, not at the moment I inspect the widget. I needed the tooltip to be dynamic.

Thing is... Textual tooltips can't be functions (which would be the obvious approach to make it dynamic); so there was no way to get this on-the-fly behaviour I wanted.

Except there was! The type of tooltip is RenderableType. So that means I could assign it an object that is a Rich renderable; that in turn means I could write a __rich__ method for a class that wraps a widget and then reports back what it can see every time it's called.

In other words, via one step of indirection, I could get the "call a function each time" approach I was after!

It works a treat too.

All of which is a long-winded way of saying I now have a print-debug-level DOM inspector tool for when I'm building applications with Textual:

If this sounds handy to you, you can grab the code too. Install it into your development environment with pip:

pip install textual-dominfo

and then attach it to your app or screen or some top-level widget you're interested in via on_mount; for example:

def on_mount(self) -> None:
    from textual_dominfo import DOMInfo
    DOMInfo.attach_to(self)

and then hover away with that mouse cursor and inspect all the things! Whatever you do though, don't make it part of your runtime, and don't keep it installed; just make it a development dependency.

The source can be found over on GitHub and the package is, as mentioned above, over on PyPI.

The idea for this one popped into my head while on the bus back from Textual Towers this evening. So after dinner and some nonsense on TV I had to visit my desk and do a quick hack.

This is textual-countdown, a subtle but I think useful countdown widget for Textual applications.

The idea is that you compose it somewhere into your screen, and when you start the countdown the bar highlights and then starts to shrink down to "nothing" in the middle of its display. When the countdown ends a message is posted so you can then perform the task that was being waited for in an event handler.

Not really a novel thing, I've seen this kind of thing before on the web; I'm sure we all have. I just thought it would be a fun idea for Textual applications too.

I envisage using this where, perhaps, an application needs to wait for an API-visiting cooldown period, or perhaps as a subtle countdown for a question in a quiz; something like that.

Anyway, if this sounds like it's something useful for your Textual applications, it's now available from PyPI and, of course, the source is over on GitHub.

textual-astare is another Textual-based Python project that I've developed in the last year and I don't believe I've mentioned on this blog. Simply put, it's a took for viewing the abstract syntax tree of Python code, in the terminal.

I've just made a small update to it this evening after someone asked for a sensible change I've been meaning to do for a while. When I first read the request I was going to look at it next week, when I have some time off work, but you know how it is when you sit at your desk and have a "quick look".

So anyway, yeah, v0.8.0 is out there and can be installed, with the main changes being:

• Updated textual-fspicker
• Updated textual
• Made it so you can open a directory to browser from the command line.
• Made opening the current working directory the default.
• Tweaked the way dark/light mode get toggled so that it's now command-palette-friendly.

I think the code does need a wee bit of tidying -- this was one of my earliest apps built with Textual and my approach to writing Textual apps has changed a fair bit this year, and Textual itself has grown and improved in that time -- but it's still working well for now.

I don't think I've mentioned it before on this blog, but some time back I decided it would be fun to use Textual to write a Mandelbrot explorer (simple Mandelbrot explorers have been another one of my favourite known problem to try an unknown thing problems). Doing it in the terminal seemed like a fun little hack. I started off with creating textual-canvas and then built textual-mandelbrot on top of that.

Not too long back I added a "command palette" to Textual (I'd prefer to call it a minibuffer, but I get that that's not fashionable these days), but so far I've not used it in any of my own projects; earlier today I thought it could be fun to add it to textual-mandelbrot.

Most of the commands I've added are trivial and really better covered by (and are covered by) keystrokes, but it was a good test and a way to show off how to create a command provider.

Having started this I can see some more useful things to add: for example it might be interesting to add a facility where you can bookmark a specific location, zoom level, iteration value, etc, and revisit later. The command palette would feel like a great way to pull back those bookmarks.

What I really liked though was how easy this was to do. The code to make the commands available is pretty trivial and, I believe, easy to follow. Although I do say so myself I think I managed to design a very accessible API for this.

There's more I'd like to add to that (the Textual command palette itself, I mean), of course; this was just the start. Support for commands that accept and prompt for arguments would be a neat and obvious enhancement (especially if done in a way that's reminiscent of how commands could be defined in CLIM -- I remember really liking how you could create self-documenting and self-completing commands in that).

All in good time...

Since quickly hacking together textual-query-sandbox a few days back, I've made a bunch of small changes here and there. While most have been cosmetic and playing with some ideas, some have also been internal improvements that should make the tool work better.

The most prominent change is one I pondered in the previous post, where I thought it might be interesting to have a small collection of playgrounds grounded together with a TabbedContent. So as of now the tool still has the original playground which had an emphasis on nested containers:

There's now a playground with an emphasis on selecting widgets within containers¹:

There's also now a playground that has an emphasis on pulling out widgets based on ID and classes:

The other change you will notice from the original post is the DOM tree shown in the bottom right corner. Note that that isn't there to show your query result (that's the bottom left panel), it's there to help picture how the DOM in the current playground hangs together, and will hopefully help in picturing the structure for when you write a query.

I sense there's still a lot of fun things I could add to this, and I'm still keen on the idea of having the playgrounds "soft coded" in some way, so people can make their own and load them up.

Another thing I want to try and work on is making the display as useful as possible. While I think it's actually pretty neat and clear, there's not a lot of space² available to show the playground and the results. Finding a good balance is an interesting problem.

For a number of reasons this is turning into a really enjoyable tinker project.