My journey to the dark side is complete

Posted on 2020-06-14 20:00 +0100 in Emacs • Tagged with Emacs • 4 min read

Ever since the whole business of "light mode" and "dark mode" really kicked off in the mobile OS world, I've been a fan of the dark modes. On both Android and iOS/iPadOS, when apps became available with dark modes, I'd switch to it. When the operating systems themselves adopted the switch, that's what I went for. As well as having a love for all things black (anyone who knows me personally will know that), I think it just looks better on mobile devices. I can't quite say why, but it just works best for me.

So, when macOS got a supported dark mode, I instantly switched it on, obviously. Then, within a day, I switched it back to light mode. Surprisingly it just didn't work for me, and I wasn't sure why. Since then I've tried living with it a few times and it's rarely lasted more than a few hours. There was something, something I couldn't quite put my finger on, that didn't sit right.

Last Monday I decided to give it another go. This time, however, I thought I'd figured it out. The "problem" was Emacs! As mentioned back in January, for as long as I've used it (so since the mid-1990s), my Emacs has always had a light background -- probably because that's how it came "out of the box" (I'm talking Emacs in graphical mode here; I started with it on OS/2 Warp and then moved to a GNU/Linux X-based desktop). I figured that the contrast between the colour scheme of Emacs, and the rest of the machine, was the issue here.

I spend most of my working day either in Emacs, or in iTerm2 -- often rapidly switching between the two. I've always run iTerm2 in the usual dark background mode with light text. So I figured the problem was having a dark OS desktop, dark terminal, and then a light editing environment. Eventually that'd feel wrong.

So I decided that, for the first time in about 25 years, I should give Emacs a go with a dark mode. Taking a quick look at popular dark themes I noticed sanityinc-tomorrow-night looked easy on the eye, so I gave that a go (actually, I gave each of the themes in that set a go, initially starting with sanityinc-tomorrow-eighties, but I finally settled on sanityinc-tomorrow-night).

To start with it didn't look too good; not because of a problem with the theme itself, but because, over time, I'd made changes and tweaks to my setup that assumed I'd be using my usual light theme. After some dabbling and tinkering and trying things out, I got it looking "just so".

Screenshot 2020-06-14 at 19.40.43.png

Having got that working, I then thought it would be nice to be able to have Emacs -- at least on restart -- adapt to me switching between dark and light mode on macOS. It turns out that detecting if macOS is in dark mode is easy enough, the command defaults read -g AppleInterfaceStyle will emit Dark if in dark mode. So, knowing that, I updated my personal package for checking things about the environment that Emacs is in to use that information:

(defconst is-a-macOS-dark-mode-window-p
  (and
   is-a-macOS-window-p
   (string= (shell-command-to-string "defaults read -g AppleInterfaceStyle") "Dark\n"))
  "Are we running in a macOS window in dark mode?")

So, as of now, my Emacs setup is such that, if I'm in graphical mode on macOS and I'm in dark mode, Emacs will use a dark theme, otherwise it'll do what it did before -- with a light background in a graphical mode and a more Borland-a-like blue background when in CHUI mode.

This seems to have made a difference. Almost one week on my work Macbook is still in dark mode, and I've switched both of my personal Macbooks, and my iMac, into dark mode too. I think it's sticking this time. Next up is to give some serious consideration to darkening the web in general. Only now am I noticing just how damn bright most of it is!


As a slight aside to the above, I've also made one other change to Emacs: I've finally dropped the display of scroll bars. In the themes I'm using they didn't look so great, appearing to be distracting. For the past 25 years I've had the scrollbars there, but never actually used them; all they've ever done is serve as a visual aid to where I am in a file. Thing is, I'm not sure I ever really pay that much attention to that either. So, as a test, I've also been running with them turned off and, so far, I'm really not noticing them been gone.

The habits we form that we convince ourselves make sense...


Some more ~/.emacs.d evolution

Posted on 2020-06-07 16:48 +0100 in Emacs • Tagged with Emacs, Lisp, Emacs Lisp • 3 min read

When I started with this version of the blog on Hashnode1, my plan was to try and write something at least once a week. I managed to do that as far as January this year, give or take, but then things got a little busy, the world got trickier, and... Well, you don't need me to tell you about that. Anyway, I'm back to write some more and, hopefully, try and keep up with writing. I feel it'll do me good.

I have been busy since January. Work has, as far as possible, carried on as normal. The only big difference being I'm back working from home. It's an odd turn of events personally given that I'd worked from home for 21 years, then (with some trepidation) went back to working in an office. Having got used to being in an office and being around people, it took some adjusting to working alone again.

So, on to the main point of this post... During that period I've spent a lot more time at my machine than I would normally, and so it was inevitable that I'd end up tinkering with my Emacs configuration. While it's true that there's hardly a week goes by where I don't tinker with some small thing, I try not to make huge changes too often. Huge changes do happen though.

A handful of weeks back was another one of those times where I made a big change.

After I burnt my original ~/.emacs file back in 2016 I've had an approach where I've tried to make things as modular as possible, and as easy to clone down to a new machine as possible. The design I came up with, especially once I moved over to using use-package, has served me well. But there was always one thing that bothered me: the files that handled the loading and configuration of packages were, in effect, still three large monolithic files. While this was better than a single monolithic ~/.emacs, it still didn't feel quite right.

Giving it a bit of thought, I decided that what I really wanted was a single directory in which I could drop lots of small .el files that would handle the loading and configuration of all sorts of packages. While it didn't gain anything concrete, the idea felt tidier. Tidy is good. Feeling like the code smell is good, is good too.

First though I needed a way to load multiple files, ideally within a whole directory hierarchy, without needing to know in advance what files would be there. After a little bit of tinkering I settled on this bit of code that revolves around the use of directory-files-recursively:

(let ((source (expand-file-name "init.d/packages.d/" user-emacs-directory)))
  (when (file-exists-p source)
    (cl-loop for use in (directory-files-recursively source (rx ".el" eol))
             do (load (file-name-sans-extension use)))))

Simply put, this code finds every file in and below ~/.emacs.d/init.d/packages.d/ whose name ends in .el (note the use of a regular expression, using rx to create it), removes the extension from the name, and loads it with load (dropping the extension means load can decide if it wants to load the compiled version of the code, or the source, depending on what's available).

After writing that I could then start to populate ~/.emacs.d/init.d/packages.d/ with lots of smaller files that handled the loading of packages. In some cases there's a file just for one package, in other cases there's a file that handles a logical grouping of packages (where there are obvious direct dependencies, or where one package is designed to extend the other, etc). For now I've decided to break things down into three directories that map to the three monolithic files I had before:

Screenshot 2020-06-07 at 16.36.28.png

However, I'm not totally wedded to this design and I may change this as time goes on.

While I've not spent any time properly testing it out, I've also not really noticed any obvious impact on startup speed. However, this tends not to be a real concern for me. I seldom start Emacs anyway. I always have it running and use emacsclient as my editor in most places so files open instantly in the running version of Emacs.


  1. Note from future me: we're now back over on blog.davep.org of course. 


dnote.el - A wrapper for the dnote CLI

Posted on 2020-01-19 16:35 +0000 in Emacs • Tagged with Emacs, Lisp, Emacs Lisp • 3 min read

Late on last year I stumbled on an article about dnote. Annoyingly, I can't recall now where I saw it, but I made a reminder to look at it over my Christmas break.

Dnote looked like a tool that would fill a hole I had in how I work. When it comes to making notes about things, and keeping things for future reference, I use a few tools, each one being just right (for me) for the job in question. I use Evernote to track documents and other household type things. I use Keep to make notes about stuff I need to remember short-term (say, the size of a space in my bedroom that I want furniture to go in) and also to record notes while in meetings at work. I use Journey to keep a journal about... anything, really. Finally, I use Pinboard to keep hold of URLs I might want to go back to (I also use it to create a to-read list).

Amongst all of this, however, I felt I was missing something for keeping track of things that don't really fall into any of the categories above. Mostly this would be work-based or hacking-based things that are short and sweet but I don't always use enough to easily remember. I wanted just the right tool that would let me ferret away useful one-liners, remind myself of obscure switches that get used once or twice a year, etc.

After reading up on dnote it seemed pretty clear that this was just such a tool.

After getting back to the office at the start of this month I decided to make use of it and see how it went. My idea was simple: I'd record any "TIL" stuff that I might want to remember in the future, as well as recording things I need now and again but can't always remember.

So far it's working quite well. I like that it has a simple CLI. I like that it's got a backend that you can use to sync between different machines. I like that it's got a web interface that's mobile-friendly. I like that it's Free Software and so you can host your own server if you wish.

I found I liked it enough that, of course, I felt the need to start a simple Emacs wrapper for the CLI.

At the moment dnote.el is designed to be a simple capture system. There are commands for capturing a one-liner (entered in the Emacs mini-buffer), for capturing the content of the current buffer, and for capturing freshly-entered multi-line text, entered in a buffer that uses markdown-mode. There's also a command for syncing notes if you have configured dnote to talk to a backend.

What I don't have right now is the ability to navigate and view notes. So far I've not really felt the need for that because the CLI approach works so well. Longer-term though I can see my tweaking this and adding in commands for searching, viewing, editing and deleting notes.

But, for now, if you've not had a look at dnote, I'd highly recommend having a play and seeing if it makes sense for you. And, if it does, and you're an Emacs user, perhaps dnote.el will be useful too?


Where I live and work

Posted on 2020-01-11 14:17 +0000 in Coding • Tagged with Emacs, shell • 2 min read

It's no surprise that I spend a lot of time in Emacs. Especially when I'm developing software, either for work or for personal fun, most of my time is time spent in Emacs. While I do obviously flit over to Chrome, and mostly do my CLI stuff in iTerm2 (I really like eshell but it just can't replace a good terminal for me), I spend a lot of time looking at Emacs.

Here's what my Emacs looks like:

Screenshot 2020-01-11 at 13.49.04.png

Key elements for me are as follows:

Light background

Something I've never really got on with when it comes to code editing is dark themes and dark backgrounds. I find it too much of an eye strain. Oddly, I tend to prefer dark themes everywhere else, but not when it comes to working in Emacs. The theme I use is the built-in adwaita theme.

Less boring mode line

I make use of powerline to make the mode line a bit less boring-looking. While the colour scheme is such that it's kept in line with the light look, the style is nice in that it sort of matches the style of prompt I use in my shell.

Screenshot 2020-01-11 at 14.05.39.png

Full screen

I always run Emacs as a full-screen application, then splitting it into different tiled windows using its own internal window handling. This is something I've done from way back when I got started with my first GNU/Linux desktop machine, and still like to do on macOS.

I also run Emacs as a server and then use a little wrapper around emacsclient to open files (both locally and remotely) from the command line in that Emacs session.

Comfortable eshell when I need it

Although I say above that I generally don't use eshell, preferring to use a full-featured terminal application, in combination with fish, I do sometimes dip into eshell for quick things. So of course I have that configured to feel comfortable too.

Screenshot 2020-01-11 at 14.10.07.png

I do this easily thanks to eshell-git-prompt.


nuke-buffers.el -- Tidy up open buffers in Emacs

Posted on 2019-12-14 13:17 +0000 in Emacs • Tagged with Emacs, Lisp, Emacs Lisp • 2 min read

Both at work and at home I use Emacs by keeping a copy running all the time, and use emacsclient to open files inside it (including on remote machines thanks to a bit of ssh and heavy use of tramp -- I might write this up at some point). This works really well, but does mean I tend to build up a lot of buffers over time.

Having lots of buffers open isn't generally an issue, and if I'm working on lots of different files in a project during the course of a hacking session it's actually a good thing. But, quite often, I want to tidy up the buffer list, clearing it back to near-zero buffers open. Many years ago, when I had a "proper" tower system running 24/7, with Emacs open all the time, I'd use clean-buffer-list as part of midnight. Along the way that fell out of favour with me, likely because I drifted into using machines that had Emacs open all the time but where the machine wasn't awake all the time.

Eventually I decided to have some fun rolling my own solution, and nuke-buffers was born.

Rather than try and do things in an automated way, this was designed to be bound to a key (or two) and then be run when I wanted, being as harsh as possible about cleaning up the buffer list. Since first writing it it's worked well for me.

These days I tend to let the buffer list build up as I work on a new feature, or chase down a bug, etc. Then, once I've made the final commit for that period of focus, I'll hit the nuke-buffer key combo as the final act of confirming that I've done the job. So not only does this help tidy my Emacs session a bit, it also feels like a physical form of punctuation -- back in less sensible days, when I had some terrible habits, it would have been when I'd reach for the celebratory cigarette; buffer-tidying feels far more wholesome. ;)

The way the code works is, of course, mostly directed at how I work -- it's highly likely it wouldn't make sense for many other people. The main aim is to kill as many buffers as possible, but without disturbing anything else. The list of buffers it gathers for nuking avoids buffers that are visiting files but have unsaved content, avoids the minibuffer (obviously), avoids any "special" buffer (one that starts with a space then an asterisk), avoids the current buffer and also avoids any buffer in a list of names to avoid.

I've being using this on a daily basis for around 2.5 years now and it's done the job without ever losing me any work.


git2gantt -- Simple tool to visualise coding runs

Posted on 2019-12-08 13:44 +0000 in Python • Tagged with Python, documentation • 3 min read

At the start of this year, as part of a much bigger process to review the work that had taken place over the previous 12 months, I was asked (at work) to provide some information about how much time I'd spent on various projects. Now, for me, there's really only one project, but there's lots of different tools and libraries that I've written to support the main work I do. All of these are split into different repositories in the company-internal instance of GitLab. This meant that getting a rough idea of what I was working on and when would be easy enough -- it's all there in the commit history.

Given that this information would make up a couple of slides at most during a far bigger presentation, I wanted something that would be snappy and easy for non-developers to follow and understand. I spent a bit of time pondering some options and decided that (ab)using a gantt chart layout would make sense.

That choice was made all the more easier given that GitLab supports the use of mermaid charts within its Markdown. This meant I could quickly write some code that took the git log of each repository, turned it into mermaid code, and then render it (by hand, this was all about getting things done quickly) via GitLab.

This sounded like it could be a fun personal project. The result was some Python code called git2gantt.

As mentioned above, the output isn't anything too clever, it's just code that can be used to create a plot via mermaid. For example, running git2gannt over itself:

gantt
  title git2gantt output
  dateFormat YYYY-MM-DD

  section git2gantt
  Development: devgit2gantt20190208, 2019-02-08, 2019-02-13
  Development: devgit2gantt20190214, 2019-02-14, 2019-02-15
  Development: devgit2gantt20190303, 2019-03-03, 2019-03-04
  Development: devgit2gantt20191203, 2019-12-03, 2019-12-04

Usage is pretty straightforward: Screenshot 2019-12-08 at
13.18.12.png As you can see, it can be run over multiple repos at once, and there's also an option to have it consider every branch within each repository. Another handy option is the ability to limit the output to just one author -- perhaps you just want to document what you've done on a repo, not the contributions of other people.

Also especially handy, if you don't want to bore people with too much detail, is the "fuzz" option. This lets you tell git2gannt how relaxed you want it to be when it tries to decide how long a run of work on a repo lasted. So, perhaps, you're working on and off on a library that supports some other system you're documenting, but you might only be making changes every other day or so. With the correct fuzz value you can make it clear you were working on the library for a couple of weeks, despite there only being a commit every other day.

An example of running the output over a handful of projects would look something like this:

Screenshot 2019-12-08 at 13.34.41.png

This is one of those tools I knocked up quickly to get a job done, and haven't quite got round to finishing off fully. One thing I'd really like to do is add mermaid support directly within it, so that it actually has the option to emit plots, not just mermaid code (or, perhaps, drop the mermaid approach and use something else entirely).

Meanwhile though, if you're looking for something quick and dirty that will help you visualise what you've been working on and when for a good period of time... perhaps this will help.


Being phony, and Lispy regular expressions

Posted on 2019-12-01 16:42 +0000 in Emacs • Tagged with Emacs, Lisp, Emacs Lisp • 2 min read

While it does seem that they're a little out of fashion these days, in some circles anyway, I'm still an avid fan of make and make files. Even in environments where I don't need a Makefile to actually build anything, I'll use one (or more) to help create handy shortcuts for getting stuff done.

Looking at the main Makefile for one of my major work projects, there's 45 targets that help fire off various jobs (all of them self-documenting using a variation on an approach I read a while back).

In most cases the targets aren't real targets. That's to say, they don't build the thing they're called. They are phony targets. So, as makes sense, I make a point of marking them all as such. I follow the convention that has the .PHONY marker appear on the line before the target; this feels cleaner to me and easier to follow and maintain.

But.... I'm lazy. And I use Emacs. Typing out .PHONY foo all the time feels like far too much work. So, some time ago, I quickly threw together make-phony.el.

With this I could be really lazy. I could type out the Makefile target and then, with my cursor on it, press a key combination and have the .PHONY marker put in place.

Does it save much time? Yeah, probably not really. But it was a fun little exercise and an excuse to write a little bit of Emacs Lisp.

There's one thing I made a point of doing in the heart of this too: using rx. For anyone who doesn't know of it, think of it as a very Lispy way of writing regular expressions. I won't even try and explain it all here because others have done an excellent job already. What I will do is say this: if you're in the habit of writing some Emacs Lisp, or even tinkering with your configuration, and you find yourself writing a regular expression, consider looking at rx -- it's well worth the time to get to know it.

Slowly, as time goes on, I'm weeding out "vanilla" regular expressions from my config and code and moving over to using rx. I feel, quite rightly I think, that something like this:

(rx
 (or
  ;; Ignore hidden files.
  (group bol ".")
  ;; I never want to edit the desktop.
  (group "Desktop/" eol)
  ;; Ignore compiled files.
  (group "." (or "pyc" "elc") eol)
  (group ".egg-info/" eol)))

is much easier to write, read and maintain, than this:

"\\(^\\.\\)\\|\\(Desktop/$\\)\\|\\(\\.\\(?:\\(?:\\(?:el\\|py\\)c\\)\\)$\\)\\|\\(\\.egg-info/$\\)"

I mean, even if the regular expression above can be written in a more efficient way (and I imagine it can), as someone working in a Lisp environment, I'd much sooner write and work with the rx version.


EmacsConf 2019 videos are out

Posted on 2019-11-24 17:17 +0000 in Emacs • Tagged with Emacs • 1 min read

At the start of this month I spent a very enjoyable Saturday afternoon watching EmacsConf 2019. If you missed it, or even if you didn't, you might like to know that the announcement has gone out that almost all of the videos are now available for watching.

While I mean nothing negative about any other video, I think the two that stand out the most for me, in terms of things that grabbed my interest, were How a Completely Blind Manager/Dev Uses Emacs Every Day and Emacs as my Go To Script Language.

I seem to recall that both those talks were hit with technical issues (many talks on the day were -- part of the enjoyment for me was how the whole conference was an experiment in using Free Software to host the whole thing), but they're well worth a watch despite this.

If you have any sort of interest in Emacs I'd highly recommend dipping into the whole collection.

I really hope there's going to be a 2020 EmacsConf.


Visual evolution of ~/.emacs.d

Posted on 2019-11-23 14:42 +0000 in Emacs • Tagged with Emacs, Lisp, Emacs Lisp • 2 min read

As detailed in a blog post I wrote back in 2016, I first got into using Emacs in the mid 1990s, starting with it on OS/2 and then moving over to GNU/Linux. It's been my often-used and much-loved development environment for most of those years (I even have a couple of packages that are part of Emacs itself).

For most of that time my configuration was a single ~/.emacs file, which was around 1,000 lines in length (including comments and whitespace). It'd grown over the years, having special configuration sections for versions of Emacs I didn't use any more, and operating systems I didn't work on any more (yes, really, there were things in there specific to MS-DOS, for example). On top of that I always hand-installed packages I used -- Emacs' package management system having turned up long after I first got into using Emacs.

Then, in early 2016, I decided to nuke the whole thing and start from scratch. As mentioned above, the start of this is detailed in an older post. Another big round of changes happened round a year later -- which included the birth of delpa to manage my personal packages. A couple or so months later there was one last big round of changes, mostly killing off my enthusiastic embracing of customize and instead going back to hand-set settings, only this time done via use-package.

The full history of this can be found over on GitHub, starting with the first "throw everything away and start again" process and all the steps between then and where my Emacs configuration is now.

Which brings me to the fun part of this blog post. Earlier this week I stumbled on Gource. It's a tool that's primarily designed to visualise changes in repositories, although it can be used to visualise anything that has a tree structure and changes over time (this week I produced a video of the growth of my employer's electronic lab notebook by hooking up the Benchling API with Gource, for example). So I got curious. What did it look like as I reworked and tweaked and changed and tinkered with my Emacs configuration?

This is what it looked like:

Visit YouTube


Getting started

Posted on 2019-11-17 11:36 +0000 in Coding • Tagged with Coding, learning • 2 min read

By coincidence, in a couple of different places over the last couple of weeks, the subject of "how do I progress in leaning to program?" has cropped up. For me, I think the approaches and solutions tend to be the same for when I want to get my head around a new language: read good examples of idiomatic code, read other related materials, find a problem you care about and implement a solution (ideally something you'll directly benefit from, or at least others may benefit from). Hence the 5x5 puzzle and Norton Guide reader projects I mentioned in my previous post.

Of course, not everyone has problems that they need solving in a way that would work for this approach. So another approach I've recommended in the past is to go looking on somewhere like GitHub and find projects that promote "low-hanging fruit" issues in a way that's designed to be friendly for those who are new to development, new to contributing or new to the problem domain.

While looking for examples of this yesterday I stumbled on Awesome for Beginners. This looks like a great list and one I'm going to keep bookmarked for future reference. Now, this particular list does seem to have an emphasis on pulling in people who are new to contributing to a project rather than new to development, but it does strike me as a good place to start looking no matter where you're coming from.

I know I'm going to start having a wander around that list. It's always nice to contribute and I feel there's real personal benefit in actively solving a problem that someone else has and welcomes help with.