Posts tagged with "PyPI"

Following on from yesterday's release of BlogMore, I've been looking at some more information in the Google Search Console, which helped me uncover a couple more bugs in relation to URL generation.

This time I noticed a couple of issues, both related to the clean_urls setting. The first was that, in the recently added calendar page, all of the URLs for the links into the date-based archive weren't taking clean_urls into account. That's now fixed.

The second problem was the canonical <link> tag in the headers of the various archive pages (categories, tags, date-based): none of the URLs used in the tag were being cleaned up if clean_urls was true. That's now also fixed.

The main "problem" those two issues were causing was Google was seeing the sitemap for my blog declare one URL, but discovering different versions of the URL elsewhere; the main offending part here being the canonical URL declaration that disagreed with the sitemap.

To the best of my understanding the above fixes should clean a lot of that up.

Also in this new release is a small new feature. After cleaning up the sitemap generation in v2.12.0 I got to thinking that, perhaps, there would be occasions where a user would want to be able to add extra items to the sitemap. With this in mind I've added the sitemap_extras configuration property. With this you can declare extra URLs to drop into the sitemap, if one is being generated.

sitemap_extras:
  - /some/path/
  - /some/file.html

I don't think I have a use for this right now, I'm not sure I'll ever have a use for it, but it feels like a low-cost feature to add that could be useful to someone at some point.

Three months back I released obs2nlm, a tool that takes an Obsidian vault and turns it into a single Markdown file so it can be used as a source for NotebookLM.

Since then I've been using it a lot and it's working out really well.

Meanwhile, one of my vaults has started to creep up towards the documented word limit for a single source in NotebookLM (500,000 words). Right now it's sitting at around 75% and is steadily creeping up.

So, with this in mind, I've made a change I've been planning from the start and have added a --split option. If used, if the generated file looks like it's going to hit the word limit, a second (or more) file will be created. The naming scheme is simple enough: if you ask obs2nlm to create an output file called dirt.md and it needs to run over, it'll then create dirt-2.md, dirt-3.md, and so on. The idea then is that, rather than upload that single Markdown file as a source, you upload all of the generated Markdown files.

Given you get up to 50 sources per notebook, this should see me right for any reasonable vault. As for if it will affect the quality of the results I get when I query the notebook... that's hard to say until I find myself in that situation. If Google are to be believed it shouldn't be an issue, and the alternative is to fall foul of the limit so this seems like the only sensible solution.

I've also added a --dry-run command line switch too; this should be handy for checking how big a vault is when compared to the word limit, without actually generating any files.

Since kicking off building BlogMore and swapping this blog over to using it I've been playing with the Google Search Console. It's something I've not used in decades, but felt it was time to dip back in again and understand how it works these days.

There are two motivations for this: the first is that, when it comes to my day job, I have cause to interact with people who do use the search console a lot, and so it's worth understanding what they work with and why it matters to them. The second reason is it's a reasonable measure of how good a site BlogMore generates.

So far the results have been pretty good, and the console has helped me find oddities and things that need tidying up.

So this release of BlogMore includes a couple of changes that stem from looking at the latest updates in the console.

The first is that I've cleaned up how the sitemap.xml gets generated. I noticed that if I had any HTML inside my extras directory it was turning up in the sitemap; something I didn't intend and didn't want. So that's now fixed: only pages generated by BlogMore will appear in sitemap.xml¹.

The second is that the stats page, despite being in the sitemap, had a noindex header for some reason. That's now been fixed. The only generated page I've intentionally set up so that it isn't indexed is the search page.

Finally, there's one change unrelated to the above: I realised that if you have with_read_time set to false, the reading time stats still appeared on the stats page; that seems unnecessary and unwanted on a site that doesn't show reading times. So, as of v2.12.0, that section of the stats won't show if reading times are turned off.

After adding the streak display to the stats a couple of days back, I got a little more obsessed with knowing what sort of runs of days of posting to the blog I had. I even said in that post:

It almost makes me want to do a whole-blog-lifetime version of it, or perhaps some sort of more calendar-oriented version of the archive.

Despite saying that I fancied the idea of that calendar-type view, first off I got to thinking it would be interesting to see a table of my 10 longest streaks. So that got added and can now be found in the stats page.

Having added that, I kept thinking about the whole-blog visual view of "here's the whole time of the blog, and here are the days you posted". I did think it might be interesting to use the same style and layout as the streak display -- perhaps something that would look like my whole contribution history on GitHub that I wrote about back in 2023 -- but the problem with that is it's tricky to make it work well on all display types. I needed something that would collapse better on smaller displays.

So I decided that a more conventional calendar display might work better. While it took a bit of work to get it to really land as I wanted, it turned out pretty much how I wanted.

So now there is a with_calendar configuration option that, if set to true, will add a calendar link at the top of the site. By default it looks like this:

If it looks a little unconventional at first glance, that's because it is. I wanted something that started with the most recent month in which there's a post, and which then worked backwards. This way I can see things as a proper history. But I can also see that this might seem odd to some people. Given this, I've also added a forward_calendar configuration option that can be used (when set to true), to flip the calendar into a more normal flow.

As you might expect, the calendar links to other parts of the site: clicking on a day with a post takes you to the archive for that day, clicking on a month name where there are posts in a month takes you to the archive for that month, and the same again for a year title.

I'm pretty pleased with the result. In testing it seems nicely responsive to different display types and I'm also finding it to be yet another interesting way to discover older posts (and get a sense of when I was encouraged to post going back over the last 11 years of this particular blog¹).

One final little feature I've added is a small enhancement to the read time that can appear on each post. While it's long since been possible to decide if you want it there or not, the calculation itself has been hard-wired to the assumption that 200 wpm is the reading speed of the reader. I've now added read_time_wpm as a configuration option so you can set it to suit your own taste.

I've released an update to BlogMore, with another little straightforward addition. This time I'm revisiting the statistics page and adding a streak tracker, of sorts.

Modelled after the GitHub contribution tracker, or indeed any number of other streak trackers, it shows which days in the recent past I've blogged on, and also an indication of how many posts I've made that day.

Of course, it's not quite a full streak tracker. It's only going to show the days up to the day the site was last generated; so when a reader visits and looks, if you've not generated the site for a month, it's not going to show that you've not blogged for a month¹. The point is that if you last blog in January, come March or so the reader isn't going to see 2 months of empty days, until you regenerate the site.

So, not perfect, but good enough I think. Also it gives the reader another method of discovering posts (each cell will take them to the archive for that day, so they can read the post or posts for that day).

I've also tried to make it vaguely responsive. There are narrower date ranges as the display gets narrower. We start out at 10 months (as you can see above), then drop to 9 months:

and then dropping to 5 months once we get to mobile-type screens:

For all its flaws, I feel it's kind of fun and I like it as a new discovery tool. It almost makes me want to do a whole-blog-lifetime version of it, or perhaps some sort of more calendar-oriented version of the archive. For now though I'm going to settle with this and see if it encourages me to keep up a blogging streak.

While it isn't my intention to write posts for the sake of it, I am enjoying writing something more frequently, so this might just help keep me doing that.

After releasing blogmore.el v2.6 this morning, I noticed something about the post: the text that was marked up with <kbd> wasn't really standing out as keys. In blog posts, as in documentation, if I mention the name of a key, I like to mark it up with <kbd>. Ideally, with such markup, the styling of the page it's being used on will make it clear that it's supposed to be read as a key.

I've never put any such styling into the default styles made available in BlogMore.

So here we are with BlogMore v2.9.0, now with a bit of markup, and theme support, for keys marked up with <kbd>. So now, hopefully, if I say you should press Ctrl+F4 to make this blog look better, those keys should stand out a little better than they used to.

I've just published v2.8.0 of BlogMore to PyPI. This is a small update which addresses a bug that Andy reported.

The fix was simple enough, and is another little interesting thing to keep in mind given that BlogMore is an ongoing Copilot experiment. When I first kicked off BlogMore I let it decide which library to use to handle Markdown (I'm more used to markdown-it-py via Textual and so via Hike), and so also let it decide which extensions made most sense given the request. I've honestly never run into the idea of metadata before, only ever dealing with or caring about frontmatter¹.

On the other hand, I will say this: I was cooking dinner when the report came in; I pointed Copilot at the issue and let it figure it out. After eating, clearing things away, and general post-dinner chilling, I dropped into the repo to see what it had made of it and... it had figured the issue out and fixed it.

For about 2 and a half years now this blog has been built with Pelican. For the most part I've enjoyed using it, it's been easy enough to work with, although not exciting to work with (which I think is a positive thing to say about a static site generator).

There were, however, a couple or so things I didn't like about the layout I was getting out of it. One issue was the archive, which was a pretty boring list of titles of all the posts on the site. It would have been nice to have them broken down by date or something, at least.

Of course, there are lots of themes, and it also uses templates, so I could probably have tweaked it "just so"; but every time I started to look into it I found myself wanting to "fix" the issue by building my own engine from scratch.

Thankfully, every time that happened, I'd come to my senses and go off and work on some other fun personal project. Until earlier this week, that was.

The thing is... I've been looking for a project where I could dive into the world of "AI coding" and "Agents" and all that nonsense. Not because I want to abandon the absolute thrill and joy I still get from writing actual code as a human, but because I want to understand things from the point of view of people who champion these tools.

The only way I'm going to have an informed opinion is to get informed; the only way to get informed is to try this stuff out.

So, here I am, with my blog now migrated over to BlogMore; a project that gives me a blog-focused static site generator that I 100% drove the development of, but for which I wrote almost none of the code.

At the moment it's working out well, as a generator. I'm happy with how it works, I'm happy with what it generates. I also think it's 100% backwards-compatible when it comes to URLs and feeds and so on. If you do see anything odd happening, if you do see anything that looks broken, I'd love to hear about it.

As for this being a "100% AI" project, and how I found that process and how I feel about the implications and the results... that's a blog post to come.

I took lots of notes.

I honestly can't remember when I was first introduced to the idea of RSS/Atom feeds, and the idea of having an aggregator or reader of some description to keep up with updates on your favourite sites. It's got to be over 25 years ago now. I can't remember what I used either, but I remember using one or two readers that ran locally, right up until I met Google Reader. Once I discovered that I was settled.

As time moved on and I moved from platform to platform, and wandered into the smartphone era, I stuck with Google Reader (and the odd client for it here and there). It was a natural and sensible approach to consuming news and updates. It also mostly felt like a solved problem and so felt nice and stable.

So, of course, I was annoyed when Google killed it off, like so many useful things.

When this happened I dabbled with a couple of alternatives and, at some point, finally settled on TheOldReader. Since then it's been my "server" for feed subscriptions with me using detsktop and mobile clients to work against it.

But... I never found anything that worked for me that ran in the terminal. Given I've got a thing for writing terminal-based tools it made sense I should have a go, and so OldNews became my winter break project.

I've written it as a client application for the API of TheOldReader, and only for that, and have developed it in a way that works well for me. All the functionality I want and need is in there:

• Add subscriptions
• Rename subscriptions
• Remove subscriptions
• Add folders
• Rename folders
• Remove folders
• Move subscriptions between folders
• Mark read/unread
• Read articles (that provide actual content in their feeds)

Currently there's no support for starring feeds or interacting with the whole "friend" system (honestly: while I see mention of it in the API, I know nothing of that side of things and really don't care about it). As time goes on I might work on that.

As with all of my other terminal-based applications, there's a rich command palette that shows you what you can do, and also what keyboard shortcuts will run those commands. While I do still need to work on some documentation for the application (although you'd hope that anyone looking for an RSS reader at this point would mostly be able to find their way around) the palette is a good place to go looking for things you can do.

Plus there's a help screen too.

If themes are your thing, there's themes:

That's a small selection, and there's more to explore.

Also on the cosmetic front there's a simple compact mode, which toggles between two ways of showing the navigation menu, the article lists and the panel headers.

OldNews has been a daily-driver for a wee while now, while also under active development. I think I've covered all the main functions I want and have also shaken out plenty of bugs, so today's the day to call it v1.0.0 and go from there.

If you're a user of TheOldReader and fancy interacting with it from the terminal too then it's out there to try out. It's licensed GPL-3.0 and available via GitHub and also via PyPI. If you have an environment that has pipx installed you should be able to get up and running with:

pipx install oldnews

It can also be installed using uv:

uv tool install oldnews

If you don't have uv installed you can use uvx.sh to perform the installation. For GNU/Linux or macOS or similar:

curl -LsSf uvx.sh/oldnews/install.sh | sh

or on Windows:

powershell -ExecutionPolicy ByPass -c "irm https://uvx.sh/oldnews/install.ps1 | iex"

Once installed, run the oldnews command.

Hopefully this is useful to someone else; meanwhile I'll be using it more and more. If you need help, or have any ideas, please feel free to raise an issue or start a discussion.

In the past few months, like a lot of folk in the Python world, I've been won over by uv. When it comes to managing my own projects my journey over the past few years as been pipenv, then rye, and then when rye was killed off I finally swapped to uv (later than I should have, I realised in hindsight). At each step I've found each tool cleaner, easier to work with and more feature-rich.

There's no doubt in my mind that uv has done the most work to make installing Python-based tools (or at least PyPI-based tools) as friction-free an experience as possible.

Now I've discovered uvx.sh. The thing with uv is the person installing your application first needs to get and install uv; this site removes that friction. Now if someone wants to install obs2nlm (for example) they should be able to just do:

curl -LsSf uvx.sh/obs2nlm/install.sh | sh

and away they go (assuming they have curl installed, which is generally going to be far more likely than having uv installed).

Of course, there are the usual caveats and concerns about the "just pipe this stuff via sh trust me bro" approach, but if you are comfortable with suggesting this kind of install method it looks worth adding this to an application's installation documentation.

I think I'm going to start mentioning it as an option.