Posts tagged with "Blogging"

The job is finally done. After considering moving all the images in the blog over to WebP, and then finally getting the migration under way, I'm all done.

As I mentioned before: I've done this by hand, one post at a time, also adding missing covers as I go. The process went faster than I anticipated and I found that adding linting support to BlogMore really helped with this process. Each time I made a batch of changes I could run the linter to make sure I'd not broken any image links.

As for the result: I've brought the total size of images on the blog down from around 56MB to about 32MB, give or take (keep in mind the latter figure also includes all the WebP images I've added while blogging since I started this process). While I don't really have to worry so much about the storage costs of these images (I'm using GitHub Pages after all), overall, over time, there should be savings in the time it takes for readers to load any given page.

The new linting tool I've added to BlogMore has paid off already. While it is the case that it helped me find a couple of broken links and one or two other things to tidy, as I was working on the feature; by the time I released it, my blog was lint-free.

But last night I did a little more work on the slow migration of images over to WebP. As I've mentioned before: this is a process I'm doing by hand, one post at a time, for a couple of different reasons. The thing is, I'm in a part of my blog now where I was often posting about updates to projects I was working on (Tinboard being a good example), and the cover for all of the posts would be the same. To save having multiple copies of the cover image, all subsequent posts would point back to the first cover image¹.

So what was happening was, I'd have a cover image that got transitioned from PNG to WebP, and then the covers of a number of posts, later in time, would be broken. While I would get to them eventually, if I'd called it a day there and rebuilt my blog, those would have been published broken.

Using blogmore lint while making those changes yesterday evening alerted me to this right away.

I've finally made a proper start on the planned migration to webp for images. I did consider writing a tool that would go through and migrate the files, and update the Markdown, all in one go, but something about that makes me kind of nervous. While it wouldn't be a destructive approach (the whole blog is under version control after all), I just have this niggling feeling that I'd miss something and it would sit broken, unnoticed, for ages.

So instead I've decided to take a one-post-at-a-time approach, making the migration by hand. As well as having the benefit of letting me go slowly and check my work as I go, I can also do some tidying up of old posts. So while I do this I'm also going to tidy up obviously broken links when I notice them, and also remove embedded tweets (swapping to the simple blockquote version).

Another thing I'm doing is adding cover images where possible. I'd been running this blog for a long time before I started to use cover (it might be that I didn't start until I moved to Pelican). Since then I've tried to use it any time there's an appropriate image in a post. More recently, I added cover images to the graph view so they're even more useful now. Back-adding a cover to older posts will make them more appealing to discover in the graph because those older notes will acquire attention-grabbing thumbnails too.

One thing I wanted to do was have an easy way to keep track of where I'm up to in the migration. It's going to be a steady process that's going to take a few days, doing a few posts at a time. So to aid this I've added this to the Makefile of the blog:

cd content/extras/attachments
find -E ./ -iregex '.*\.(png|jpg|jpeg)$' | cut -d'/' -f2,3,4 | sort -u

With this I get a handy list of dates of posts that still have unconverted png or jpeg files.

Of course, for a wee while, this will not get to an empty list because I want to make sure some of the more recent posts still have their older images available as they might be in feeds out there. More recently I've only been using webp for images, so once the webp-using posts fill the main RSS and Atom feeds I can clean out the last of the bulkier images.

Ever since I kicked off the work on BlogMore I've had a renewed interest in writing on this blog (as you can probably tell from the stats and the calendar). But not just writing: also tweaking it, tidying it up, thinking about maintaining it into the future, thinking about the links and the categories and so on.

In doing so, I've also been looking at other folk who persist in keeping a blog, and especially those who maintain blogs built with static site generation tools, and in some cases I'm mildly envious of how far back some of them stretch.

When it comes to the world of blogging I was kind of late to the party. The first version was just a section of my self-developed website, hosted on www.davep.org. Don't go looking there for it now, it was long ago removed. In fact my personal website is mostly just a placeholder for what once was. The Wayback Machine still has a copy though, so I can see that the first blog post I wrote for my site was dated 2003-03-31.

I maintained this for a while, the engine for it all being some self-written PHP engine that was what could be best described as a dynamic static site (in other words it generated everything on request from underlying text files and HTML snippets because I had no wish to be faffing around with databases on a web host). Eventually though the blog side of this got to be too much trouble and I jumped over to Blogger.

I maintained that blog for quite a few years, with the first post being made in 2006 and the last in 2011. Sadly it's all quite broken now. I used to include a lot of images and, while some of them are embedded in the site itself, most were hosted on the older version of my website, as part of the photo gallery I also had there.

This all fell apart when I finally killed off the PHP version of my site and all the images were removed. Now the blog is a wasteland of broken image icons (not to mention a wasteland of broken external links -- so many of the sites I referred to back then have fallen off the net).

I hate this. I hate that thirty-something me was fired up enough to want to write stuff down and communicate to other people (and to future me) and it's all decayed. I especially dislike that the original version of my blog, now only stored on the Wayback Machine (and perhaps on a hard drive that I think is in a box somewhere in storage, perhaps also on some burnt-as-a-backup DVDs) is otherwise inaccessible. Much like I did with my original photoblog, I want to rescue this. I want to rescue all of this.

The technical challenges of teasing out the original posts from the Internet Archive and from Blogger aren't too great. Turning a bunch of HTML into Markdown isn't impossible either -- the library that I use in OldNews should do the job fine there. All that sort of work feels like a fun little challenge that will keep me amused for a few evenings.

There are two main things that cause me to pause when thinking about doing this.

The first is that some of those very old posts, as I mentioned above, link to places that don't exist and haven't existed for a long time. It raises the question: do I even care to preserve things that have no context any more?

The second is that many of the posts in the Blogger blog, as I mention, relied on images hosted on my old site. Right now I'm not actually sure where those photos are! While I took a backup of all the code and other data for www.davep.org when I did the big reboot (storing it all up on GitHub), I seem to have stripped out all of the photos. This makes sense as there was a lot of data there. Making sure I had a backup of those files feels like something I would do -- I hang on to all sorts of data -- but at the moment I can't locate them¹.

To make this work, for this to stand any chance of working, I need to pull them all back out from somewhere.

Will I do this? I don't know yet. The seed is there, the itch is waiting to be scratched. I look at the age span of this blog, and the calendar page, and think it could be really cool to really back-fill it from my older blogs. The graph might end up looking really funky.

On the other hand: am I just trying to preserve irrelevant things as a way to make work for myself (albeit "work" that is fun; after all coding is a hobby as well as a living).

On the gripping hand: if I can get the images back, a wasteland of links to sites that don't exist any more does, at the very least, provide a history of what was and is no longer.

For a good while now I've been pretty happy with the PageSpeed measurements of this blog, which in turn means I've been happy with the state of what's generated by BlogMore. I have pretty much everything that can be minified nice and minimal. At this point, the main thing that causes the speed measurement to fluctuate is image sizes.

I use a lot of PNGs on this blog. When I'm using images, they're almost always in posts that include screenshots, which in turn pretty much demand that I use a lossless format. When I take these screenshots I don't worry too much about the dimensions (within reason), and of course I don't really do anything to optimise how they'll work and appear on different display sizes. If I was to get too into that, it would add friction to writing something, and the whole point of this is to feel less friction when it comes to sitting at the keyboard.

So I've been living with the fact that some images can be pretty big. While I do make a point of using pngcrush on every image, it generally doesn't make a huge saving.

Then yesterday I read this post on Andy's blog and I suddenly realised what I had to do!

Borrowing from what Andy did, I used mogrify too, setting up this Fish abbr in my Fish configuration:

if type -q mogrify
    abbr -g mkwebp "mogrify -format webp -define webp:lossless=true -quality 100"
end

In my case, at least in the initial experiment, I decided to keep it all lossless. So far the results have been really good, cutting the image sizes down by a significant amount. For example, if I look at the images for yesterday's posts:

 90581 15 Apr 18:14 sl-overview.png
 33446 16 Apr 20:23 sl-overview.webp
392661 15 Apr 18:14 slstats-region-info.png
225392 16 Apr 20:23 slstats-region-info.webp
 36049 15 Apr 19:39 year-chart.png
 15590 16 Apr 20:23 year-chart.webp

That's a pretty reasonable saving!

So far all I've done is convert the few latest posts that make up the front page of my blog, just so I can see what impact it has. I'm getting improved load times on mobile, for sure.

There are a couple of downsides to this, of course.

1. Now I want to do the whole blog, so while I can easily go through and convert all the png files to webp, converting all the image markup in the Markdown files isn't quite so simple, and even if I do write something to automate it, I'm then going to want to review it to make 100% sure nothing has broken.
2. I can't just then remove all the png files to cut back on the space used by the generated site. The front page of the site has a feed, and all the categories have a feed each too. This means that there could be HTML out there from some of my oldest posts, referring to the png files, and just removing them will result in broken images.

Overall though, it might be worth doing at some point soon. Meanwhile, from now on, I think I'm going to replace my pngcrush step with a mkwebp step and just use webp instead of png now.

I guess I'm all modern now!

This was a nice find yesterday: I think I came across it when someone I follow on Mastodon boosted a post from the account related to the site; it's a site called powRSS. The concept is pretty simple: collect links to all sorts of small blogs on all sorts of topics, and then provide a honking great discovery feed/pool. You can read more about the idea on their about page.

For sure, this sort of thing isn't exactly novel: those of us of a certain age will fondly remember the fun of webrings and other similar initiatives, not to mention feed aggregation sites where you could discover trending blogs or see what your friends were reading, and all that. But, to some degree, that fell out of favour and/or the limelight when social media got really popular.

So with this in mind it's good to see people still providing such sites. I've added this blog to it and I'll be diving in there now and again to see if there's anything new I should be following.

It'll be fun to populate OldNews with more things to read.

Following on from the rescue of my photoblog, followed by the rescue of the original incarnation, I've been thinking that I should get into the habit of posting the odd image here and there again, when the desire takes me.

The problem is the workflow. The point and purpose of the last two incarnations was that I just had to take image, manipulate image, post image and then lots of other things would happen. As I've mentioned in the posts linked to here, and others here on this blog, it's much harder these days to achieve that seamless and frictionless workflow we used to enjoy during the rise and the peak of "Web 2.0".

So yesterday I decided on an approach that, while it's neither seamless nor frictionless (yet), it feels like it will work. So here's the plan:

• Take photo
• Manipulate photo¹
• In Apple Photos add a description to the image
• Export JPEG file to a Photoblog Inbox folder in iCloud Drive
• Later, when I'm at my desk, import the image into the photoblog
• Publish the updated blog

So, yeah, more manual steps are required, at least for the moment. I suspect though that if this works well I might be able to increase the automation involved and do it in such a way that I'm in control of the steps and services (or at least I'll be able to do it in a way that I can swap out a step should whatever service being be lost).

As an aside, for anyone who might have been following along with my experiments with GitHub Copilot: the import tool I link to above was written from a prompt. This, for me, feels like the ideal use for such a tool. Writing that script was going to be a chore, I wanted to get on with the bigger picture thing, so asking Copilot to write it while I attended to other things felt sensible and useful.

I'll likely tidy up the code soon, but for now what was produced works and it let me try all of this out in the time I had available on an otherwise busy Sunday afternoon.

At the end of yesterday's post I said I might see if I can rescue the original photoblog from its backup on WordPress. This was the first photoblog I played with, posting to the long-dead Posterous between 2009 and 2013.

So, yesterday evening, I did an extract of the full feed from WordPress, and also asked for a full backup of all the media. I then fired up Emacs and rattled out some Python code that would marry up the two sets of data and add to the photoblog repository. It took a little bit of working out; it seems that every post had two entries in the feed: a parent and a child entry. I've no clue why that's the case; I didn't really care to get too deeply into it.

Soon enough seen-by.davep.dev was updated with an extra 1,833 posts! So now the categories on that blog site are broken down into Seen By Me 1 (the original) and Seen By Me 2 (the second incarnation).

Sadly, for the first blog, tagging wasn't really much of a thing so the tag cloud hasn't grown too much.

But, finally, I've got both the photoblogs back up and hosted somewhere I can point to, and I'm fully in control of their content. While it is hosted on GitHub Pages I've done this in a way that it would be pretty easy to move elsewhere; this is down to the fact that it's a simple static site built with BlogMore.

Since mid-2023 my photoblog has been broken. As I mentioned at the time, the issue was that this second incarnation of the blog had started life as a proper mashup of some web tools, and the heart of it was Twitter.

It all started to fall apart when Twitter got its new owner, and APIs became expensive, and other tools would not or could not work with it any more, and then it really fell apart when I finally nuked my account.

So since then the blog has been sat about, unused and unloved, with a lot of broken links and images.

Thankfully, though, the pipeline that I had going had been designed with this sort of problem in mind: part of what I had going also made a backup of the photos I took to Google Drive and to Google Photos. So when I got to a point the other day where BlogMore was usable I decided I should rescue the photos and rebuild the blog.

After downloading the full feed of the Blogger.com-hosted blog, I threw together some Python code that took the data (thanks to feedparser for helping with that), matched up the posts with the images I had in Google Drive, slugged the names, wrote some Markdown and copied some images, and I had the source of a fresh blog.

The result of all of this can be seen up on seen-by.davep.dev.

I strongly suspect this is going to remain a pretty static site. At the moment I've given no thought whatsoever as to how I might have this populate in the way the old version of it did. Quite simply the old version was:

1. Take photo
2. Post to Twitter with a specific hashtag
3. Have IFTTT notice this and create the blog post, make backups
4. ...
5. Profit?

I suppose, this time around, I could have something monitor my Mastodon account, or my pixelfed account, and then trigger a similar process; but then that would need something akin to IFTTT running and interacting with GitHub and creating the Markdown and kicking off a build process and...

Eh, maybe one day. For now, though, I'll be happy that I've rescued this particular incarnation of my photoblog and then think about if and how I carry on with something similar in the future.

Meanwhile... this has got me thinking. The original blog is backed up on WordPress. It's been sat there, all sad and neglected, ever since Posterous disappeared. I wonder if I can export all the data from there and mash it into this new version...

For about 2 and a half years now this blog has been built with Pelican. For the most part I've enjoyed using it, it's been easy enough to work with, although not exciting to work with (which I think is a positive thing to say about a static site generator).

There were, however, a couple or so things I didn't like about the layout I was getting out of it. One issue was the archive, which was a pretty boring list of titles of all the posts on the site. It would have been nice to have them broken down by date or something, at least.

Of course, there are lots of themes, and it also uses templates, so I could probably have tweaked it "just so"; but every time I started to look into it I found myself wanting to "fix" the issue by building my own engine from scratch.

Thankfully, every time that happened, I'd come to my senses and go off and work on some other fun personal project. Until earlier this week, that was.

The thing is... I've been looking for a project where I could dive into the world of "AI coding" and "Agents" and all that nonsense. Not because I want to abandon the absolute thrill and joy I still get from writing actual code as a human, but because I want to understand things from the point of view of people who champion these tools.

The only way I'm going to have an informed opinion is to get informed; the only way to get informed is to try this stuff out.

So, here I am, with my blog now migrated over to BlogMore; a project that gives me a blog-focused static site generator that I 100% drove the development of, but for which I wrote almost none of the code.

At the moment it's working out well, as a generator. I'm happy with how it works, I'm happy with what it generates. I also think it's 100% backwards-compatible when it comes to URLs and feeds and so on. If you do see anything odd happening, if you do see anything that looks broken, I'd love to hear about it.

As for this being a "100% AI" project, and how I found that process and how I feel about the implications and the results... that's a blog post to come.

I took lots of notes.