Posts tagged with "Coding"

I've been pretty quiet on here since I last wrote about how I'd done a further revamp of my Emacs config, so I thought that subject would be a good reason to write another blog post.

It'll be a mostly short one, and one to muse over something that's been bugging me for a while now: my decision to lean heavily on customize to set all sorts of settings.

Initially, when I nuked my original config over a year ago, it seemed to make a lot of sense. Let all the tweaks and set values "hide" in a file of their own and try and keep the hand-edited config files as small and as clean as possible. Recently though I've got to thinking that this obscures too much, hides too much detail, and removes the ability to actually document what I'm doing and why. It also does make it tricky to adapt some settings to different platforms or even environments on a single platform.

Another problem I've run into is this: when I made the second round of changes and decided to lean heavily on use-package, I soon ran into the minor issue of some packages not making sense, or even being needed, on some platforms (stuff that's useful on my macOS machines isn't always useful on my Windows machines, that sort of thing). While use-package can handle this easily thanks to the :if keyword, I'm still left with the fact that package-selected-packages still gets populated.

Having package-selected-packages contain a list of installed packages likely makes sense if you're using just the Emacs package system and you're not doing the installing with use-package and :ensure. But with use-package and :ensure I feel like I've got far more control over things and can adapt what gets installed when depending on which Emacs I'm running where.

But, because I'm syncing my ~/.emacs.d/.custom.el to all my machines too, any use-package that has a :if to not bother using a package has little effect because the package still ends up being listed/loaded/seen as part of the installation.

Ideally, I think, I'd like to be able to have package-selected-packages held in its own file, or I'd only ever use ~/.emacs.d/.custom.el for local stuff (and so stop syncing it).

Starting today I'm going about a process of moving as much as I can out of ~/.emacs.d/.custom.el and into hand-edited files. In some respects I guess I am going back to how I used to manage Emacs configuration, but this time it's not a massive monolithic file-of-Lisp, it's neatly broken down into sensible sections and it's also biased towards a "grab and config this package" approach.

Meanwhile, I've not seen any good discussions online about customize vs "hand-edit", which strikes me as a little odd as it feels like the perfect "religious issue" for people to have endless disagreements over. I guess, over the next couple or so weeks, I'll find out if switching back was a good idea.

Just under a year ago I decided to totally rewrite my GNU emacs config. As I wrote at the time, it'd been following me around all sorts of machines since the early 1990s, starting life on an OS/2 Warp machine and travelling via MS-DOS, GNU/Linux, Windows and, these days, macOS.

The changes I made last year have served me really well, but there were two related issues with it that bothered me a little: the fact that I was maintaining a local library of elisp code in the repository and, worse still, I was storing the packages I'd installed from elpa and melpa in the repository as well.

While this did mean it was pretty easy for me to start up a new installation of emacs on a machine -- all I had to do was clone the repo and run up emacs -- I wasn't happy with the duplication involved. I didn't like holding code in my .emacs.d repo that was already held in package archives.

The solution I saw was in two parts:

1. Get some of my code, that might be useful to others, into melpa.
2. Somehow sort my own package archive for my personal code.

Over the past week or so I've worked on this approach. It initially started with me tackling item 1 above: I tidied up and submitted obfusurl.el, protocols.el, services.el, thinks.el and uptimes.el. This was a really helpful process in that it allowed me to brush up on my elisp and emacs knowledge. It's a good 15+ years since I last wrote any significant elisp code and things have moved on a little in that time.

Having done that I'd managed to move a handful of my own packages out of my local library of code, and so out of my .emacs.d repo, but it left me with the problem of what to do with the rest of it.

That's when I discovered package-x and:

,----[ C-h f package-upload-buffer RET ]
| package-upload-buffer is an interactive compiled Lisp function in
| ‘package-x.el’.
|
| (package-upload-buffer)
|
| Upload the current buffer as a single-file Emacs Lisp package.
| If ‘package-archive-upload-base’ does not specify a valid upload
| destination, prompt for one.
`----

(plus package-upload-file too, of course). This meant I could, in effect, start my own personal package archive and look at tackling issue 2 above.

This did give me one small problem though: how and where would I host the archive? I did consider hosting it on a DigitalOcean droplet, but that felt a little like overkill for something so simple. And then I realised: GitHub Pages! All I needed to do was keep the package archive in its own repo (which I would have done anyway) and then make the whole repo the source for a GitHub Pages site. A quick test later and... it worked!

So, by this point, I'd farmed some of my code off to melpa, and now had the rest of it in "delpa" (which I'd called my personal archive). I could now use the emacs package management system to install third party packages and also my own.

But I was still left with one issue: I was still holding the installed packages inside my .emacs.d repo by way of ensuring that all machines were in sync in terms of what was installed. Now I needed to work out how to solve that.

Around this time, as luck would have it, @tarsius had suggested I look at a package called use-package by @jwiegley. This was the bit I was missing.

With use-package I would be able to declare which packages I needed, how they'd be installed and, most important of all, it could be set to handle the fact that the package wasn't even installed. If a package is requested and there is no local install use-package is smart enough to get the emacs package system to install it.

So, given that, all I need to do was create a startup file that would declare the packages I use and I'd have a setup that should, once I'd cloned .emacs.d, self-install.

Except... yeah, one more issue. use-package isn't part of GNU emacs yet so I'd need a method of getting it to auto-install so it could then handle everything else. As it was that was as easy as adding this to the start of my init.el.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Make sure the package system is up and running early on.
(require 'package)
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/"))
(add-to-list 'package-archives '("delpa" . "http://blog.davep.org/delpa/"))
(package-initialize)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Bootstrap `use-package'
(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))

With that in place I was able to nuke all my config on a machine, clone a fresh copy of .emacs.d (having now ceased tracking and storing the installed packages in that repo), run up emacs, wait a few moments and then find that everything was installed and ready to use.

Perfect!

My .emacs.d is now a lot smaller than it was before and, I think, even easier to maintain. Right now I think I'm very close to the ideal emacs config that I wanted to create when I did the complete rewrite a year ago.

Like many programmers, I have a couple of "Hello, World" projects that I've carried with me over the years. One is 5x5 (which has been used to get to grips with things as diverse as the Palm Pilot and GNU emacs). Another is Norton Guides database readers.

I've made Norton Guides tools that have allowed web servers to serve guides (w3ng), that have allowed you to convert guides to HTML (ng2html), that have let you read guides on OS/2 and GNU/Linux (eg) and also have let you read guides in Microsoft Windows (weg). It's a problem I know fairly well and one where I know the solution well enough so I can concentrate on learning the new language or environment.

Recently I wanted to get to grips with some "pure" ES6 coding while also getting to know node.js. A new version of the Norton Guide code, written for this environment, seemed like a good thing to do.

And so jsNG was born.

At its core is a library of code for opening and reading data from Norton Guides databases. While I doubt it's good ES6 code, or even good node.js code, it's been very useful in giving me a fun problem to solve and it'll carry on being something I'll tweak and tinker with by way of trying new things out.

On top of this I've built a handful of tools for working with Norton Guides databases. The most useful one at the moment (the others are more in the "test the library" than the "make something handy with the library" category) is ngserve. This is designed as a simple Norton Guides database HTTP server.

When run, you give it a list of guides to serve:

and it does the right thing. It has a small number of command line options that help configure what it does:

Possibly the most useful are the ones that let you change how it handles "higher" DOS characters and, if you don't like the default colours and stuff, the option that lets you point to your own style sheet (note for now you'll need to host the stylesheet somewhere else -- ngserve won't serve it for you; I'm aiming to change that in some way in the near future).

jsNG does have a fairly basic design compromise at its heart. In the very early version I started out using the async functions for opening and reading the guides. This got very tedious very quickly and I could see that it was going to make for a very messy library with a very messy interface. While it might not be in the spirit of node.js programming I decided to go with the sync version of the file IO functions and code up the core library based around this.

This approach also means that I took another leap that I never have done with Norton Guides before: rather than doing the traditional thing of keeping an open handle into them and reading direct from the file as you navigate the guide, I simply read it all into a buffer in one go and keep it in memory. This is a "guides are small, memory is cheap, things will go faster" approach.

It does mean that when you load up a load of guides into ngserve they're all sat in memory. The upside of this is that things should be a lot faster and the code is a lot easier to follow (I think). To put this in some perspective: I have a directory here that contains 110 Norton Guides files. They total 36M in size. If that seems like a lot of stuff to hold in memory... remind me how much is being used by your web browser so you can look at some hilarious kittens. ;)

Anyway, that's where I'm at with it right now. The code is mostly settled and mostly tidy. I need to write up some documentation for it (and so I need to take a look at good JavaScript documentation tools) and perhaps tinker with ngserve a little more. I'd also like to do a new version of ng2html with this -- a version that makes it far easier to control the style of the output. I'm also tempted to do a CLI-based reader in pure ES6; something similar to EG or WEG.

All in good time.

The other day I stumbled on a tip about Evernote. It was a snippet of information in a bigger post on the Evernote blog so I saved it to Evernote (obviously) to take a look at later.

The tip was that Ctrl-Cmd-B in Evernote (on the Mac) will format a body of text as source code. While I'm not in the habit of using Evernote to store code, not even snippets of code (that's something far better served by Gist), it seemed like something worth committing to memory.

This morning, while finally in front of a suitable machine, I took a look. Sure enough, there's the menu option.

Handy! So I created a test note with some code in it so that I could see how it formatted it. I was curious to see if it just did simple fixed text or if it offered options to highlight various languages (I didn't hold out much hope for the latter, but it was worth a look).

That's when it got odd.

After I created a note and went to format some code, the menu option disappeared!

From what I can tell, once it's gone, there's no way to get it back. At least, not until you actually close down Evernote and start it up again.

Frustratingly, none of this is the case in the Windows version. While the key combination is different there, the menu option is available and stays available.

The version of Evernote I have (on my iMac and my Macbook) is 6.9.2. The version number on Windows is 6.4.2 (after checking for updates -- I'm going to guess that the Mac and Windows version numbers don't match on purpose).

Goodness knows what's going on here. All I can imagine is that it has something to do with a thread I found on Evernote's support forum that suggests that code block formatting is some sort of test/beta feature and can only be enabled via a settings option that isn't available via the version downloaded from the Apple App Store.

So, at some point, I guess I'm going to have to uninstall Evernote from the Macbook and the iMac, reinstall from the Evernote website itself, and try this again. All of which seems a bit silly when the menu option is there and visible when I run Evernote up!

Edit to add: Sure enough, removing the App Store version and installing the version from Evernote's own website, and then going into preferences and enabling the option, sorted it. It's still really odd they'd promote the facility via the blog and not mention it, and also very odd that the option would always show until the first time you're in a position to use it, and then it'd disappear.