Posts tagged with "BlogMore"

As I've written about a few times in the last week or so, the journey with AI-based coding tools has hit an interesting time when it comes to prices, quotas, usage, availability and all that. Having come into all of this via a place where it was a flat fee, and where I didn't really need to think about input tokens and output tokens and so on, I'm pretty ignorant of what that all means in terms of scale. If I'm looking at a new tool and I see prices and/or quotas for in/out tokens, it means nothing to me. I can't relate to it. I've never had to care about it.

While using Gemini CLI to quickly make a change to BlogMore this morning, I was reminded that at the end of a session it does tell me this:

Seeing that got me thinking: is there a way to get the total usage for all of my sessions, or at least the sessions that have still been retained (I'm guessing they expire after a wee while)? After a little bit of searching I found ccusage. That looked exactly like the sort of thing I was after.

Now, this is only going to be good for Gemini (it says it supports Copilot too, but it seems to be failing to find any Copilot sessions), but it should give me a feel for what my token usage looks like.

I work on BlogMore on two different machines: the MacBook Air and also a Mac Mini I have in my office. Here's all of the available token usage data I can get out of the Air:

And also the same for the Mac Mini (which gets used less frequently for this sort of thing):

In both cases I've removed a couple of columns to make the tables fit better. The first was the model name (varying between gemini-3-flash-preview and gemini-3.1-pro-preview), the second was Cache Create (which was always 0 all the way down).

From what I can see, it would appear that these two tables do cover my increasing use of Gemini CLI for doing work on BlogMore (the first intensive use being back around the 5th of this month, if I recall correctly). So this would seem to be a reasonably informative way to view things.

All of which is to say, over a roughly three week period, while getting things done, I've used getting on for 20,000,000 input tokens, and around 600,000 output tokens (presumably I do also need to be keeping the 114,300,000 cache read tokens in mind too). With this in mind I might now be able to make more sense of the pricing I see for various tools.

Now that we're near the end of the free or cheap GitHub Copilot party, I thought it might be interesting to look at how much BlogMore has "cost" me to build, and what it would have cost under the proposed new pricing structure that is coming in next month. While I've looked at the comparison for last month, I've not looked at the whole period I've been seriously using it.

So, for this review, I'm looking at all the data I can pull out of GitHub for the months of February, March, April and May of this year. Development of BlogMore started back in February and, while it hasn't been 100% the cause of my use of Copilot premium requests, it's been almost all of it. For the purposes of this review I'm just going to take the approach that all I worked on was BlogMore.

Remember that, even when I had free access, I had a maximum of 300 premium requests per month. Once I lost free access I had the same number of requests for $10 a month.

Here's how those months broke down:

So, give or take, something that I've actually spent $30.00 on could have, at best, cost me $185.51. That's assuming that the "cost" of the models I was using stays the same. You can see that the costs have risen already in that the predicted price from February, where I used 83% of my premium requests, is a touch under half the cost for this month, where I've used just 11%. From what I can see in the raw data, it's down to some models suddenly being considered more expensive (perhaps I was doing something that just consumed more tokens, I'm not 100% sure if I'm honest, but I don't recall anything that seemed like harder work).

Who knows what the real costs will be come June.

Now, technically, the actual cost under the new regime could or should be $156, because it would be 4 lots of the $39.00/month plan, which would better cover that use¹. Again though, that's assuming the actual cost of using whatever models remains pretty stable. It also assumes that I'd want to spend that much each month, and that I would be correctly anticipating that I'd need that much.

Also, this isn't even the total cost of getting this project done. As I've written recently: I've been using Gemini CLI more this month, and while the usage there is a flat cost, until now, that's changing too.

Now, of course, these aren't the only games in town. I could "go to the source" and just get a sub for Claude Code or something, and as Tim pointed out over in the Fediverse, something like Cursor does a lot of this and is just $20/month. Which all sounds fine, but what happens when those fleeing GitHub Copilot or Gemini CLI/Antigravity head over to something like Cursor? Is it sensible to expect the pricing to stay the same²?

I guess, at this point, I'm just mulling over the same issue time and again, but from different angles. It does seem clear to me, though, that in less than 4 months, in my experiment of "what happens if I use agents to develop a Free Software tool I want?", the market has gone from being entirely reasonable to pretty much unjustifiable from a price point of view.

Following on from the previous release, which was all about trying to get a big PageSpeed Insights win through image optimisation, I'm chasing some more validation from that site by trying to squeeze just a little more performance out of the code that BlogMore generates.

BlogMore v2.25.0 has the following changes to allow tinkering in ways that might speed things up a touch, depending on the nature of the blog:

CSS bundling -- Every page generated by BlogMore pulls in at least these three CSS files: style.css, code.css and fontawesome.css (or their minified versions if minify_css is turned on). While this separation of concerns sits well with me, while it feels like the elegant way of doing things, there is the issue that it requires 3 trips back to the server to get base styling for any given page¹.

So with this new version, if you set bundle_css to true, those three files are included and delivered as a single bundle.css (or bundle.min.css). This saves a couple of requests.

Theme helper inlining -- the lesser of the two main changes. There is some JavaScript that's part of each page that helps with theme switching and also provides the code to toggle the header display on mobile-sized screens. It's not a lot of code, but it is another file that has to be fetched. If inline_theme_js is set to true, this code will be included in the <head> of every single page generated for the site.

I suspect I'm going to leave this one off, but it's there if it's helpful to anyone (and also does let me experiment more with PageSpeed measurements).

Optimised logo -- one image that got left out of the work to optimise images was the site logo. While an optimised version of the image was created, no HTML was generated to make use of it. With this release, if optimise_images is true, <picture> will be used for this too.

With those shameless performance-measurement changes aside, there are a couple more changes in this release. The first is that the markup for the site title (that appears below the logo, if you have one) has been changed away from using a <h1> tag. The SEO gods frown on multiple <h1>s on a page and given the "main" title of any page is also a <h1>, this meant there were always 2 such tags. Now just the main title will be marked up this way; the site title becomes a <div> with appropriate styling to maintain the existing look.

Finally, this release fixes a small bug in the search index. It was being created with escaped HTML entities in any text that came out of fenced code blocks. From now on any text that goes into the search index is unescaped.

As always: if a blog-oriented static site generator that is all about Markdown sounds like your thing, check out the installation instructions and give it a go.

Quite a few weeks ago now -- I think it was around the time I started work on blogmore.el and got the new MacBook Air -- I remember sitting in a cafe in Edinburgh and via Mastodon having a conversation with Andy about tweaking better results out of PageSpeed Insights. I seem to remember him correctly observing that one of the big hits on the performance score was the size of images, and also the format, and that some SSG engines would go to the trouble of converting to the likes of WebP and/or generating different sizes that are appropriate to different screens, that sort of thing.

I can't quite remember where we left it, but I think it was considered more work than was worth worrying about, and perhaps swapping all images on our blogs to WebP would solve most of the issues.

For a couple of different reasons, late last week, I decided it was time to play with the problem. For some reason I've been pretty cautious with this PR. I planned it out last Friday night, kicked off work on it on Saturday morning, and have then been tinkering and changing it and testing it and iterating over it all weekend. Something about the nature of the change made me want to go very slowly with this. I think it was an unease about messing with the images that would get served, the nature of the new tags that would get emitted, the fact that there would be even more HTML tinkering going on, the possible complexity of maintaining the cache... lots of things to consider and this is supposed to be a nice, simple, unfussy site generator.

Anyway, I've just released v2.24.0 with this feature added. It's off by default, and is turned on by setting optimise_images to true. Then, when you build your blog, each PNG, JPEG or WebP image will be converted into one or more WebP images stored below static/images/optimised. How many are made for each image will depend on how image_widths is set. The physical size of each image (and how the image looks) can be affected by image_quality.

This does have two very obvious effects:

1. It will result in your generated site being quite a bit bigger, if you have lots of images.
2. It will result in the build time taking much longer.

The first issue is something I can't do anything about; it is what it is. The second issue, however, is something that can be dealt with. Given I've just made a release that speeds up build times, this would be a huge step backwards. So with this in mind, as the optimised images are created, a cache of them is also created in BlogMore's cache directory. This, again, does mean that more space is taken on your local storage to build your site, but it also means that repeated builds will remain fast.

If you run into problems or need space back, don't forget you can easily clear the cache.

So what's the result of all of this? Is it worth the effort? Well, to be sure, before I upgraded the version of BlogMore that I build this site with, I measured its performance.

After upgrading and rebuilding, here is how the same home page measures up.

I was genuinely surprised by the difference. The settings I used were:

optimise_images: true
image_quality: 95

and, of course, almost all the images on this site are now WebP anyway. I think I was expecting it to have a small impact, but even having those WebP images turned into stepped sizes seems to have a very measurable effect.

I'm going to be keeping a close eye on how this works for the next few days. As I say, I've tested this as much as possible and gone over the code as carefully as time has allowed. If this feature does break something I hadn't anticipated I can always just turn it off again anyway. Meanwhile though, the improvement on mobile does seem genuinely worth it.

Following on from this morning's initial experiment, I think I'm settling on a winner. Rather than be annoying and have you scroll to the bottom to find out: it's Gemini CLI. Here's how I found the process played out, and why I'm settling for one over the other.

Gemini CLI¶

Initially this was an absolute mess. After letting it initially work on the problem, the resulting code didn't even really run. The first go, and the three follow-up prompt/result cycles that followed, all resulted in code that had runtime errors. I'm pretty sure it didn't even bother to try and do any adequate testing. This is odd given I've generally seen it do an okay job when it comes to writing and running tests.

Once I had the code in a stable state, with all type checking, linting and testing passing, it still didn't work. No matter how I tried to use the new facility it just didn't make a difference. No images were optimised. In the end I dived into the code, with the help of its attempt at debugging (it added print calls to try and get to the bottom of things -- how very human!), diagnosed what I thought was the issue (it was looking in the wrong location for the files to optimise), told it my hypothesis and let it check if I was right. It concluded I was and fixed the problem.

Since then I've had a working implementation of the initial plan.

Once that was in place it's been a pretty smooth journey. I've asked it questions about the implementation, had my concerns set to rest, had some concerns addressed and fixed, improved some things here and there, added new features, etc.

All of this has left me with 18% of my daily quota used up. While I think this is the highest I've ever got while using Gemini CLI, it still feels like I got a lot of things done for not a lot of quota use.

GitHub Copilot¶

Initially I thought this had managed to one-shot the problem. Once it had finished its initial work the code ran without incident and produced all the optimised files. Or so I thought. Doing a little more testing, though, it became clear it was only optimising a subset of the images and it didn't seem to be producing the actual HTML to use the images.

On top of this it didn't even follow the full plan that was laid out in the issue it was assigned. For example: once I'd got it doing the main part of the work, it became apparent that it had pretty much ignored the whole idea of using a cache to speed this process up. I had to remind it to do this.

At one point I switched from the in-PR web interaction with Copilot, and used the local CLI instead. When I ran that up it warned me that I was already 50% of the way through some sort of rate limit and this wouldn't reset for another 3 hours. I think I was about 40 minutes into letting it try and do the work at this point.

After a bit more testing and follow-up prompts, I got to a point where I had something that looked like it was working; albeit in a slightly different way from how Gemini CLI did it (the Copilot approach was writing the optimised images out to the extras directory, mixing them in with my own images; Gemini opted for having a separate directory for optimised images within the static hierarchy).

At this point I will admit to not having carefully reviewed the code of either agent; that's a job still to do. But while Gemini got off to a very rocky start, with a bit of guidance it seemed to arrive at an implementation I'm happy with, and one that seems to be working as intended. While it didn't anticipate all the edge cases, when I asked about them it easily found and implemented solutions for them. Moreover, the fact that I could do all of this and confidently know the "cost" made a huge difference. Copilot seems to generally approach this like a quota or rate limit should be a lovely surprise that will destroy your flow; Gemini has it there and in front of you, all the time.

As for the general idea that I'm working on: I think I'm going to implement it. Weirdly I'm slightly nervous about building the blog such that it won't be using the images I created, but I also recognise that that's a little irrational. Meanwhile I'm very curious about the impact this might have on the PageSpeed measurement of the blog. While it's far from horrific, image size optimisation and size declaration seem to be fairly high on the things that are impacting the performance score (currently sat at 89 for the front page of the blog, as I type this).

The other thing that gives me pause for thought about merging this in, and then subsequently using it, is that I've just finished migrating all images to webp, and so saving a lot of space in the built version of the blog. Generating all the responsive sizes of the images eats that up again. With this feature off, the built version of the blog stands at about 84MB; with it on, this rises to 133MB. That extra 49MB more than eats up the 24MB saving I made earlier.

On the other hand: storage is a thing for GitHub to worry about, what I'm worrying about here, and aiming to improve, is the reader's experience.

I'm going to sit on this for a short while and play around with it, at least until I get impatient and say "what the hell" and run with it.

Given I'm almost certainly going to drop GitHub Copilot starting next month, I'm using Gemini CLI more and more for BlogMore. Yesterday evening, I used it to plan out an idea for a change to the application. Now that I've migrated all images to WebP, I thought it might be interesting to look at the idea of having a responsive approach to images. This is something I don't know a whole lot about (never having needed to bother with it before), but it also happens that I need to read up on this anyway for something related to the day job; given this, it felt like a good time to experiment.

Together with Gemini CLI a plan was created.

This morning, over second coffee, I've kicked off the job of implementing it and, honestly, Gemini CLI is really struggling. It "implemented" the change pretty quickly, within minutes, but it just plain didn't work. Since then I've had it iterate over the issue four times and now it's struggling to make it work at all. It's still beavering away on this as I type, and consuming daily quota at a fair rate too.

So, while I still have GitHub Copilot, this feels like a good point to play them off against each other at least one more time. Having saved the plan Gemini wrote last night as an issue, I've assigned it to Copilot (using Claude Sonnet 4.6). As I type this, I have Gemini racing to get this working in a terminal window behind Emacs, meanwhile there's Claude doing its thing in GitHub's cloud.

It'll be interesting to see if Copilot manages to one-shot this, for sure Gemini is far off a one-shot implementation.

I wasn't quite planning on making a new release of BlogMore so soon after the previous version, but I had a couple of ideas that I wanted to add, and then also got a nifty request too; so here we are: we have v2.23.0.

The first couple of changes relate to the cache. In the previous release I added a cache of the FontAwesome metadata, which in turn means that a cache directory is being created. I felt it would be fair and useful to provide a command that both lets the user know where the cache lives, and to also remove it. So now BlogMore has a cache command with two sub-commands:

• location: tells you where the cache directory is located
• clear: removes the cache directory

Also, now that we have a cache directory, it makes sense to use it a bit more to squeeze even more time out of the build process. So starting with this release, per content directory, the various icons that are created for the site are cached. This means that if the source image doesn't change, for each subsequent build there's no conversion and resize for every variation. This saves a good fraction of a second, making the build of my blog feel noticeably quicker.

Finally, earlier today, Andy asked if it would be possible to have the BlogMore serve mode auto-reload any page being viewed in a browser, when the site is regenerated. It was something I'd considered myself a couple of times so that was a good reason to finally look into it. Not knowing how this could be achieved¹, I prompted Gemini for an idea, stressing I wanted a solution that didn't disturb a generated site; it came up with a convincing solution. I let it run at it and, along with a few changes of my own, it seems to be working a treat.

This, of course, now makes me want to squeeze even more time out of the build process.

As mentioned a couple of days ago, I've been toying with finding areas of improvement in respect to the performance of BlogMore. Until now, for good reasons, I've not really paid any attention to how fast (or slow) BlogMore is when it comes to generating my blog. While it's never been blindingly fast, it's always been fast enough and I was more keen on making it work right. So for a good while the focus has been on well-formed output, stuff that keeps the crawlers happy, that sort of thing.

But now that I'm in a place where new features aren't really so necessary, it does feel like a good point to find any easy wins in speeding up the code. I think it's gone well.

BlogMore v2.22.0 contains quite a few internal changes that speed up some core parts of site generation. Many of the things identified by Gemini, back when I first kicked this process off, have been done. The amount of Markdown->HTML conversion work has been vastly reduced, which has had a pretty big impact on all sorts of things. There's also caching of the FontAwesome metadata¹ which should save a fair bit of time on slower connections. I did avoid the whole business of parallel processing as I dabbled with this near the start of the project and I could not wrangle a win out of that at all; given how much of a win I've had with these changes, I doubt that would change (it could conceivably make things worse).

So, how much faster is it? Roughly, based on my tests, a site generates in about 1/4 of the time it did before. On my M2 Mac Mini my blog builds in under 3 seconds; with v2.21.0 it took around 13 seconds. In my case that's with all the optional features of BlogMore turned on.

Naturally this work has touched on a lot of internals of the code, and made significant changes to the generation pipelines of lots of different pages and features. I've done my absolute best to compare² the output of v2.21.0 and v2.22.0 and I can't see any significant differences³. When trying out v2.22.0 I would suggest paying just a little extra attention to the result, to be sure you're happy that nothing has changed.

As mentioned recently, Andy did a nifty bit of testing of BlogMore to measure the performance hit of each of the optional generation features. Performance is something I haven't really spent much time thinking about; I cared more about how the result looked than I did about how quickly a site got generated.

I seem to remember that, early on, I did have a bit of a play with trying to get Copilot to tackle the idea of parallel generation of parts of the site, but I didn't seem to get any sort of win out of it so I closed the PR and gave up on the idea.

Now though, especially having added the elapsed time of each step to the progress report during generation, I am curious about what wins there are. One that came to mind right away, and I have been considering dealing with for a wee while, is the fact that every generation downloads the FontAwesome metadata. Caching that would be a simple win, for example.

But, if I'm going to do this, it makes sense that I stick to the spirit of this project and get one or more agents to tackle the problem. So I asked Gemini CLI to look over the codebase and give me some big easy wins. I made clear I didn't want it to do any work yet, I just wanted it to report back what it could find.

The result made a lot of sense. Here's what it said:

Performance Optimization Recommendations for BlogMore¶

I have conducted a comprehensive review of the blogmore codebase and identified several areas where measurable and significant speedups can be achieved, particularly for larger sites.

1. Backlink Snippet Extraction (Highest Impact)¶

The most significant bottleneck is in src/blogmore/backlinks.py. The build_backlink_map function iterates through every internal link in every post. For each link, it calls _extract_snippet, which re-parses the entire Markdown content of the source post to generate a plain-text snippet.

• The Issue: If a post has 20 internal links, it is parsed 20 times just for backlinks.
• The Win: Modify the logic to insert unique markers for all links in a post and perform a single Markdown-to-plain-text conversion per post.

2. Redundant Markdown Parsing for Metadata¶

The Post object in src/blogmore/parser.py has properties like description and reading_time that are accessed frequently (on the post page, index pages, RSS feeds, search index, etc.).

• The Issue: Post.description calls extract_first_paragraph, and Post.reading_time calls calculate_reading_time. Both of these functions perform a full Markdown conversion every time they are called because they are not cached.
• The Win: Use functools.cached_property on these fields. Additionally, these should be calculated by extracting text from the html_content that has already been generated during the initial parse, rather than re-parsing the raw Markdown.

3. Sequential Processing¶

The generator currently processes everything sequentially in a single thread.

• The Issue: For a site with hundreds or thousands of posts, the initial Markdown parsing and the subsequent Jinja2 rendering are strictly serial, leaving multi-core CPUs mostly idle.
• The Win: Use multiprocessing (e.g., ProcessPoolExecutor) to parallelize the parsing of Markdown files and the rendering of HTML pages. Since these tasks are CPU-bound and independent, this would scale almost linearly with the number of CPU cores.

4. Asset Management Inefficiencies¶

The AssetManager in src/blogmore/generator/assets.py performs some operations that could be optimized:

• FontAwesome Metadata: It downloads icon metadata from GitHub on every build to optimize CSS. This can be cached locally to avoid network latency.
• Icon Generation: It regenerates favicons and Apple touch icons on every build. This should be skipped if the source icon hasn't changed.

5. Minor Algorithmic Improvements¶

• Navigation Lookups: In PageGenerator.generate_post_page, the code uses all_posts.index(post) to find the current post's position for prev/next links. This is an O(N) operation called N times, leading to O(N^2) behavior. Pre-calculating a mapping would make this O(1).
• Markdown Instance Reuse: Several utility functions (markdown_to_plain_text, extract_first_paragraph) create a fresh markdown.Markdown instance on every call. Reusing a single instance (and calling .reset()) or caching the instance creation would reduce overhead.

Summary¶

The "Big Wins" are definitely optimizing backlink snippet extraction and implementing parallel parsing/rendering. Together, these would likely reduce generation time for large sites by 60-80% on modern hardware.

At first glance they all seem pretty sensible, and I like that it seems to have found some really obvious big improvements, and also the little wins that might have a cumulative effect. I strongly suspect the parallel generation suggestion is again going to be a non-win (and so for now I'm going to pass on it), but most of the other suggestions seem doable and worth trying out.

I think what I will do, rather than work through them all in one PR, is tackle one at a time. So, over the next few days, expect to see some PRs turn up and get merged (hopefully), as I try and tease some speed wins out of the code.

The new linting tool I've added to BlogMore has paid off already. While it is the case that it helped me find a couple of broken links and one or two other things to tidy, as I was working on the feature; by the time I released it, my blog was lint-free.

But last night I did a little more work on the slow migration of images over to WebP. As I've mentioned before: this is a process I'm doing by hand, one post at a time, for a couple of different reasons. The thing is, I'm in a part of my blog now where I was often posting about updates to projects I was working on (Tinboard being a good example), and the cover for all of the posts would be the same. To save having multiple copies of the cover image, all subsequent posts would point back to the first cover image¹.

So what was happening was, I'd have a cover image that got transitioned from PNG to WebP, and then the covers of a number of posts, later in time, would be broken. While I would get to them eventually, if I'd called it a day there and rebuilt my blog, those would have been published broken.

Using blogmore lint while making those changes yesterday evening alerted me to this right away.