Introduction

On Twitter, a few weeks back, @itsBexli asked me how I go about setting up Python for development on macOS. It's a great question and one that seems to crop up in various places, and since I first got into using macOS and then subsequently got back into coding lots in Python it's absolutely an issue I ran into.

With my previous employer, while I was the only developer, I wasn't the only person writing code and more than one other person had this issue so I eventually wrote up my approach to solving this problem. That document is on their internal GitLab, but I thought it worth me writing my personal macOS/Python "rules" up again, just in case they're useful to anyone else.

I am, of course, not the first person to tackle this, to document this, to write down a good approach to this. Before and after I settled on my approach I'd seen other people write about this. So... this post isn't here to try and replace those, it's simply to write down my own approach, so if anyone asks again I can point them here. I feel it's important to stress: this isn't the only way or thoughts on this issue, there are lots of others. Do go read them too and then settle on an approach that works for you.

One other point to note, which may or may not make a difference (and may or may not affect how this changes with time): for the past few years I've been a heavy user of pipenv to manage my virtual environments. This is very likely to change from now on, but keep in mind that what follows was arrived at from the perspective of a pipenv user.

So... with that admin aside...

The Problem

When I first got back into writing Python it was on macOS and, really early on, I ran into all the usual issues: virtual environments breaking because they were based on the system Python and it got updated, virtual environments based on the Homebrew-installed Python and it got updated, etc... Simply put, an occasional, annoying, non-show-stopping breaking of my development environment which would distract me when I'd sat down to just hack on some code, not do system admin!

My Solution

For me, what's worked for me without a problem over the past few years, in short, is this:

1. NEVER use the system version of Python. Just don't.
2. NEVER use the Homebrew's own version of Python (I'm not even sure this is an issue any more; but it used to be).
3. NEVER use a version of Python installed with Homebrew (or, more to the point, never install Python with Homebrew).
4. Manage everything with pyenv; which I do install from Homebrew.

The Detail

As mentioned earlier, what I'm writing here assumes that virtual environments are being managed with pipenv (something I still do for personal projects, for now, but this may change soon). Your choices and mileage may vary, etc... This is what works well for me.

The "one time" items

These are the items that need initially installing into a new macOS machine:

Homebrew

Unless it comes from the Mac App Store, I try and install everything via Homebrew. It's really handy for keeping track of what I've got installed, for recreating a development environment in general, and for keeping things up to date.

pyenv

With Homebrew installed the next step for me is to install pyenv. Doing so is as easy as:

$ brew install pyenv

Once installed, if it's not done it for you, you may need to make some changes to your login profile. I'm a user of fish so I have these lines in my setup. Simply put: it asks pyenv to set up my environment so my calls to Python go via its setup.

Plenty of help on how to set up pyenv can be found in its README.

Once I've done this I tend to go on and install the Python versions I'm likely to need. For me this tends to be the most recent "active" stable versions (as of the time of writing, 3.7 through 3.10; although I need to now start throwing 3.11 into the mix too).

I use this command:

$ pyenv install --list

to see the available versions. If I want to see what's available for a specific version I'll pipe through grep:

$ pyenv install --list | fgrep "  3.9"

This is handy if I want to check what the very latest release of a specific version of Python is.

The "Global" Python

When I'm done with the above I then tend to use pyenv to set my "global" Python. This is the version I want to get when I run python and I'm not inside a virtual environment. Doing that is as simple as:

$ pyenv global 3.10.7

Of course, you'd swap the version for whatever works for you.

When Stuff Breaks

It seems more rare these days, but on occasion I've had it such that some update somewhere still causes my environment to break. Now though I find that all it takes is a quick:

$ pyenv rehash

and everything is good again.

Setting Up A Repo

With all of the stuff above being mostly a one-off (or at least something I do once when I set up a new machine -- which isn't often), the real "work" here is when I set up a fresh repository when I start a new project. Really it's no work at all. Again, as I've said before, I've tended to use pipenv for my own work, and still do for personal stuff (but do want to change that), mileage may vary here depending on tool.

When I start a new project I think about which Python version I want to be working with, I ensure I have the latest version of it installed with pyenv, and then ask pipenv to create a new virtual environment with that:

$ pipenv --python 3.10.7

When you do this, you should see pipenv pulling the version of Python from the pyenv directories:

$ pipenv --python 3.10.7
Creating a virtualenv for this project...
Pipfile: /Users/davep/temp/cool-project/Pipfile
Using /Users/davep/.pyenv/versions/3.10.7/bin/python3 (3.10.7) to create virtualenv...
⠙ Creating virtual environment...created virtual environment CPython3.10.7.final.0-64 in 795ms
  creator CPython3Posix(dest=/Users/davep/temp/cool-project/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/davep/Library/Application Support/virtualenv)
    added seed packages: pip==22.2.2, setuptools==65.3.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /Users/davep/temp/cool-project/.venv
Creating a Pipfile for this project...

The key thing here is seeing that pipenv is pulling Python from ~/.pyenv/versions/. If it isn't there's a good chance you have a Python earlier in your PATH than the pyenv one -- something you generally don't want. It will work, but it's more likely to break at some point in the future. That's the key thing I look for; if I see that I know things are going to be okay.

Conclusion

Since I adopted these personal rules and approaches (and really, calling them "rules" is kind of grand -- there's almost nothing to this) I've found I've had near-zero issues with the stability of my Python virtual environments (and what issues I have run into tend to be trivial and of my own doing).

As I said at the start: there are, of course, other approaches to this, but this is mine and works well for me. Do feel free to comment with your own, I'm always happy to learn about new ways!

As mentioned yesterday, I'm about to start working at Textualize, and building Open-source software is important to the company. Will -- the CEO -- is all about building in public. If you follow him on Twitter you'll notice that his Python coding adventure tweets actually outnumber is cooking tweets!

As someone who has long been a supporter of and fan of Free Software and Open-source software, and has made some small contributions along the way, I've also always made a point of building my own tools in public. In most cases they're things that are likely only helpful to me, but some are more generally useful. The point being though: it's all there in case it's helpfull to someone else.

Which means that, as much as possible, when I'm writing code, I write it as if it's going to be visible in public and someone else is going to be reading it. I try and make the code tidy. I try and comment it well. I try (but don't always manage for personal projects) to fully document it. The important thing here being that someone coming to the code fresh should be able to follow what's going on.

Against that background, and having just gone through the process of handing off almost 5 years of work to someone else as a left an employer, I got to thinking: we should always "build in public", even if it's in private.

When I started with my previous employer, and even to the day I left, I was the only software developer there. I worked with a team who wrote code, but being software developers wasn't what they did. Bioinformaticians and machine learning scientists have other things to be doing. But, as I wrote my code, I wrote every line assuming they, or some other developer down the line, would be reading it. Pretty much every line was written for an audience I couldn't see and didn't fully know. This, as mentioned above, meant trying to keep the code clean, ensuring it was commented in helpful ways, ensuring the documentation was helpful, and so on.

But it wasn't just about the code. Any non-trivial system will have more to it than code. We had an internal instance of GitLab and I tracked all of my work on there. So, as I planned and worked on new features, or went on bug hunts, I'd document the process in the issue tracker. As much as possible I'd be really verbose about the process. Often I wouldn't just open an issue, go work on it, and then mark it closed; as I worked through the issue I'd add comment after comment under it, documenting my thinking, problems, solutions, cite sources when looking something up, that sort of thing.

The whole process was an act in having a conversation with current or future team members if they ever needed to look; with future me (really, that helped more than once -- we all have those "that the hell was I thinking?" moments); with any developer(s) who took over from me in the future.

I did all this as if I was broadcasting it in public on Twitter or on GitHub, etc. It was in private, of course, but I approached it as if it was in public.

There were always three main reasons for this, I felt:

1. Accountability. At any moment someone who I worked with could review what I was doing and why I was doing it; it was an invitation to anyone curious enough to talk with me about what I was building and how I was building it.
2. Continuity of support for unplanned reasons. Life happens, sometimes you may, unplanned, never be available at work again. I never wanted to leave my employer in a position where picking up from such an event was a nightmare.
3. Continuity of support for planned reasons. It was possible, and it became inevitable, that I'd move on to something else. If that was to happen I wanted to be sure that whoever picked up after me would be able to do so without too much effort.

In the end item 3 seemed to really pay off. When it came time for me to hand over my work to someone else, as I left, the process was really smooth and trouble-free. I was able to point the developer at all the documentation and source code, at all the issues, and invite them to read through it all and then come back to me with questions. In terms of time actually spent talking about the main system I was handing over I'd say that 4 years of work was handed over with just a few hours of actual talking about it.

It remains to be seen if it really paid off, of course. If they get really stuck they do have an open invitation to ping me a question or two; I care enough about what I designed and built that I want it to carry on being useful for some time to come. But... I like to think that all of that building in public, in private, will ensure that this is an invitation that never needs to be called on. I like to think that, if something isn't clear, they'll be able to check the code, the documentation and the issue history and get to where they need to go.

Just over five years ago I got a message from my then employer to say I was going to be made redundant after 21 years working for them. After the 3 month notice period the final day came. Meanwhile, I found something new that looked terrifying but interesting. In the end it was less terrifying and way more interesting than I imagined it would be. It was fun too.

But... (there's always a but isn't there?)

In the four and change years I've been there the company got bought out, and then the result of that got bought up. As I've mentioned before I'm generally not a "big company" kind of person; in all my years I've found that I'm happier working in a smaller place. After a couple of buyouts my employer had gone from being 10s of people in size to 100s of people in size (and technically 10s of 1,000s of people in size depending on how you look at it).

This change in ownership and size meant the culture became... well, let's just say not as friendly as you tend to enjoy when it's a smaller group of folk. On top of that I was starting to notice that my efforts were making less of an impact as things got bigger, and I started to feel like my contributions weren't really relevant any more. There were some problematic things happening too: undermining of efforts, removal of responsibilities without consultation or communication, that sort of thing. Plus worse. There's little point in going into the detail, but it's fair to say that work wasn't as fun as it used to be.

That felt like a good time to start to look around. If work makes you feel unhappy and you can look around... look around.

Thing is, I wasn't sure what to look for. I was in the comfortable position of, unlike last time, not needing to find something, so I could take my time at least. Over the course of the last year I've spoken to many different companies and organisations, some big (yes, I know, I said I don't like big places -- sometimes what's on offer deserves a fair hearing), some small, but none of them quite said "this feels like me". In some cases the whole thing didn't have the right vibe, in others the industry either didn't interest me, or felt uncomfortable given my personal values. In one particular case a place looked interesting until I checked the CTO's socials and OMG NO NO NO AVOID AVOID (that was a fun one).

Then I saw Will McGugan saying he was hiring to expand Textualize. This caught my interest right away for two good reasons.

I can't remember how long I've been following Will on Twitter; I likely stumbled on him as I got back into Python in 2018 and I also remember noting that he was a Python hacker just up the road from me. We'd vaguely chatted on Twitter, briefly, in that "Twitter acquaintance" way we all often do (I remember one brief exchange about fungus on The Meadows), and he'd seemed like a good sort. A small company run by a "good sort" kinda person felt like a damn good reason.

The second reason was Textual itself. I'd been watching Will develop it, in open, with great interest. I had (and still have) a plan to write a brand new CHUI-based (okay fine TUI-based as the kids like to say these days!) Norton Guide reader, all in Python, and Textual looked like the perfect framework to do the UI in. The chance to be involved with it sounded awesome.

Now, I said two reasons, but there's also a third I guess: Will's pitch for applying to Textualize felt so damn accessible! I'm on the older end of the age range of this industry; for much of my working life as a developer I've worked in isolation from other developers; while I first touched Python in the 90s, I've only been using it in anger since 2018 and still feel like I've got a lot to learn. Despite all these things, and more, saying "aye Dave this is beyond you" I felt comfortable dropping Will a line.

Which resulted in a chat.

Which resulted in some code tinkering and chatting.

Which resulted in...

Something new.

So, yeah, as of 2022-10-10 I'm on yet another new adventure. Time for me to really work on my Python coding as I work with Will and the rest of the team as part of Textualize.

Or, as I put it on Twitter a few days ago: I'm going to be a Python impostor syndrome speedrunner!

It's five years ago this month that I bought myself my first macOS (then OS X) device. After many years of having a Windows machine as my daily driver, which was also my work machine (I worked from home), I decided it was high time that I returned to having a Unix-a-like system on my desk too. For a decade or so, starting in the later-90s, I'd had a GNU/Linux desktop. I still had a Windows desktop (until a couple of years ago most of my work was on DOS and Windows), but thanks to the wonders of a KVM, and later an X server that ran on Windows, my personal hacking was done on a GNU/Linux desktop.

But as things moved around, priorities changed, as life moved on, the GNU/Linux boxes got retired and never quite replaced. Eventually, in 2015, I found myself with the means and desire to recover that sort of setup. Long story short, after a lot of reading up and weighing up options I decided that the best option for a desktop Unix was... an iMac!

I loved it. Sure, there were lots of little things on the surface that were different or annoying or just plain not as cool as the Mac fans would tell you, but under the hood I found what I needed: a Unix CLI with all the things I knew well. And, of course, it ran GNU Emacs just fine; that was the really important thing for me.

Pretty much right away I decided that it might be fun to learn the tools necessary to develop native Mac apps, and perhaps even iOS apps. I downloaded XCode, bought a book, and started working through it. Having got that book, I decided it might be interesting to own an iOS device too. So, sort of needing an MP3 player, and having no wish to get an iPhone, I got myself an iPod Touch. So I was all set to devour the Swift book, write some stuff for OS X, create an iOS app or two, and... life happened. Stuff cropped up that distracted me from taking that further and I never really returned to working through the book.

Fast forward to now and that initial iMac and iPod purchase spiralled a wee bit. Next after the iPod was an iPad Mini, when my Nexus 7 was starting to show its age and it was obvious that Google wasn't going to produce any more good Android tablets. Then, when I needed a very portable Unix-a-like machine for trips between where I was living and Edinburgh, I got myself a MacBook Air. Since then the iPod Touch has been replaced once, as has the iPad Mini. I now also own an iPad and a MacBook Pro. Unless Apple screw up and turn Macs into something unusable for developers (there are rumours), I imagine I'll be using Apple devices for some time to come now.

And then, last month, having finally got frustrated with where Google were going with Android and the Pixel series, I jumped ship to the iPhone 11.

As of right now I'm in a situation where I'm all about the Apple ecosystem regarding hardware and operating systems (including for my work machine), all of which is there to support my heavy use of the Google ecosystem (actually, the one bit of Google hardware I still lean on heavily is the Google Home -- I have 3 around my home).

So... given all of that, I thought it was time to look at returning to learning Swift, with a view to writing some native macOS and i(Pad)OS stuff. I soon realised that the book I'd bought back in 2015 was rather out of date. It covers Swift 1.2 -- we're now up to 5.2! Given this, and given I've forgotten pretty much everything I'd read at the time, I decided I should start again from scratch.

This weekend I've started reading my way though iOS Programming Fundamentals with Swift. While this obviously has an emphasis on iOS, I'm already finding that the first part of the book is a really great introduction to the Swift language in general. The pace seems just right, and the way topics are grouped makes it easy enough for me to skip over what's obvious (I don't need to know what objected-oriented programming is, and what the difference between a class and an object is, etc) and read up on the detail of this particular language when it comes to general concepts I know (knowing the differences between a class, struct and enum in the language is important, for example).

I've yet to write a line of code, but I'm fine with that. The book is spending a lot of time introducing the language before encouraging you to fire up XCode, and I'm okay with that. I'm never a fan of being asked to write out code that I can't properly follow -- that just makes stuff look like magic when it's far more educational to know what's going on. What I am finding is I'm making lots of notes that are either "oh, yeah, this is cool, I like this idea!" or "WTF are you kidding me?!?". Which is really nice -- it's always great to learn a new language that's a bit different from what you normally use.

My plan then, over the next few weeks, it to keep at this and hopefully document my journey. I think I'd like to write a short series of TIL-type posts; nothing too long, just some new thing I read or discovered and my reaction to it. So, if you happen to follow this blog, I apologise in advance for any Swift-spam.

You have been warned. ;-)

By coincidence, in a couple of different places over the last couple of weeks, the subject of "how do I progress in leaning to program?" has cropped up. For me, I think the approaches and solutions tend to be the same for when I want to get my head around a new language: read good examples of idiomatic code, read other related materials, find a problem you care about and implement a solution (ideally something you'll directly benefit from, or at least others may benefit from). Hence the 5x5 puzzle and Norton Guide reader projects I mentioned in my previous post.

Of course, not everyone has problems that they need solving in a way that would work for this approach. So another approach I've recommended in the past is to go looking on somewhere like GitHub and find projects that promote "low-hanging fruit" issues in a way that's designed to be friendly for those who are new to development, new to contributing or new to the problem domain.

While looking for examples of this yesterday I stumbled on Awesome for Beginners. This looks like a great list and one I'm going to keep bookmarked for future reference. Now, this particular list does seem to have an emphasis on pulling in people who are new to contributing to a project rather than new to development, but it does strike me as a good place to start looking no matter where you're coming from.

I know I'm going to start having a wander around that list. It's always nice to contribute and I feel there's real personal benefit in actively solving a problem that someone else has and welcomes help with.

Today is a fairly significant day for me. For the past 21 (and a bit) years I've worked for the same company. I know that, for lots of people in my industry, that's a hell of a long time to be in one position. It was a setup that served me pretty well; the company was small (I'm not much of a big company person), I generally got to dictate the development tools and direction, and I also got to work from home.

The working from home part was especially helpful as other life events involved me and happened around me. The job I've been in has seen a marriage come and go, and a move from Hampshire to Lincolnshire, and then from Lincolnshire to Midlothian. There's been a few adventures along the way.

But, three months ago, I got a call to say that I was to be made redundant. This sucked, obviously. It also appeared pretty stupid on the part of my employer: a company that is based around software had decided it no longer wanted an in-house software developer. A bold, and I think unwise, choice. From this point on they've decided to only go with contract developers and only contract developers from overseas (or so the story goes).

As it goes, things have turned out okay. Today is my last day with my old employer and, as I'm sat here typing this out while having some lunch, I'm actually glad that it's coming to a close and that I can get to move on.

I have a new employer, and will be starting in my new position during the first week of January. It'll be very different. It'll all be different. Not only will I be back working in an office (one where jeans and t-shirts are the norm, thankfully!), I'm also going to be working in an industry (as a developer still, of course) that I have no background in and no real knowledge of. It's going to be a seriously exciting challenge.

Over the next couple of weeks I've got a fair amount of background reading (and video watching) to be doing. This isn't so much that I can get on with the job of development (although there will be a good bit of that to do too -- who doesn't want to be trying to understand new options for development?), it's more about understanding the language of the industry I'll be in and so that I can understand the needs of my users.

Goodbye fashion retail and all related things. Hello genetics and... well, I don't know, I'm going to find out. :)

I've been pretty quiet on here since I last wrote about how I'd done a further revamp of my Emacs config, so I thought that subject would be a good reason to write another blog post.

It'll be a mostly short one, and one to muse over something that's been bugging me for a while now: my decision to lean heavily on customize to set all sorts of settings.

Initially, when I nuked my original config over a year ago, it seemed to make a lot of sense. Let all the tweaks and set values "hide" in a file of their own and try and keep the hand-edited config files as small and as clean as possible. Recently though I've got to thinking that this obscures too much, hides too much detail, and removes the ability to actually document what I'm doing and why. It also does make it tricky to adapt some settings to different platforms or even environments on a single platform.

Another problem I've run into is this: when I made the second round of changes and decided to lean heavily on use-package, I soon ran into the minor issue of some packages not making sense, or even being needed, on some platforms (stuff that's useful on my macOS machines isn't always useful on my Windows machines, that sort of thing). While use-package can handle this easily thanks to the :if keyword, I'm still left with the fact that package-selected-packages still gets populated.

Having package-selected-packages contain a list of installed packages likely makes sense if you're using just the Emacs package system and you're not doing the installing with use-package and :ensure. But with use-package and :ensure I feel like I've got far more control over things and can adapt what gets installed when depending on which Emacs I'm running where.

But, because I'm syncing my ~/.emacs.d/.custom.el to all my machines too, any use-package that has a :if to not bother using a package has little effect because the package still ends up being listed/loaded/seen as part of the installation.

Ideally, I think, I'd like to be able to have package-selected-packages held in its own file, or I'd only ever use ~/.emacs.d/.custom.el for local stuff (and so stop syncing it).

Starting today I'm going about a process of moving as much as I can out of ~/.emacs.d/.custom.el and into hand-edited files. In some respects I guess I am going back to how I used to manage Emacs configuration, but this time it's not a massive monolithic file-of-Lisp, it's neatly broken down into sensible sections and it's also biased towards a "grab and config this package" approach.

Meanwhile, I've not seen any good discussions online about customize vs "hand-edit", which strikes me as a little odd as it feels like the perfect "religious issue" for people to have endless disagreements over. I guess, over the next couple or so weeks, I'll find out if switching back was a good idea.

Just under a year ago I decided to totally rewrite my GNU emacs config. As I wrote at the time, it'd been following me around all sorts of machines since the early 1990s, starting life on an OS/2 Warp machine and travelling via MS-DOS, GNU/Linux, Windows and, these days, macOS.

The changes I made last year have served me really well, but there were two related issues with it that bothered me a little: the fact that I was maintaining a local library of elisp code in the repository and, worse still, I was storing the packages I'd installed from elpa and melpa in the repository as well.

While this did mean it was pretty easy for me to start up a new installation of emacs on a machine -- all I had to do was clone the repo and run up emacs -- I wasn't happy with the duplication involved. I didn't like holding code in my .emacs.d repo that was already held in package archives.

The solution I saw was in two parts:

1. Get some of my code, that might be useful to others, into melpa.
2. Somehow sort my own package archive for my personal code.

Over the past week or so I've worked on this approach. It initially started with me tackling item 1 above: I tidied up and submitted obfusurl.el, protocols.el, services.el, thinks.el and uptimes.el. This was a really helpful process in that it allowed me to brush up on my elisp and emacs knowledge. It's a good 15+ years since I last wrote any significant elisp code and things have moved on a little in that time.

Having done that I'd managed to move a handful of my own packages out of my local library of code, and so out of my .emacs.d repo, but it left me with the problem of what to do with the rest of it.

That's when I discovered package-x and:

,----[ C-h f package-upload-buffer RET ]
| package-upload-buffer is an interactive compiled Lisp function in
| ‘package-x.el’.
|
| (package-upload-buffer)
|
| Upload the current buffer as a single-file Emacs Lisp package.
| If ‘package-archive-upload-base’ does not specify a valid upload
| destination, prompt for one.
`----

(plus package-upload-file too, of course). This meant I could, in effect, start my own personal package archive and look at tackling issue 2 above.

This did give me one small problem though: how and where would I host the archive? I did consider hosting it on a DigitalOcean droplet, but that felt a little like overkill for something so simple. And then I realised: GitHub Pages! All I needed to do was keep the package archive in its own repo (which I would have done anyway) and then make the whole repo the source for a GitHub Pages site. A quick test later and... it worked!

So, by this point, I'd farmed some of my code off to melpa, and now had the rest of it in "delpa" (which I'd called my personal archive). I could now use the emacs package management system to install third party packages and also my own.

But I was still left with one issue: I was still holding the installed packages inside my .emacs.d repo by way of ensuring that all machines were in sync in terms of what was installed. Now I needed to work out how to solve that.

Around this time, as luck would have it, @tarsius had suggested I look at a package called use-package by @jwiegley. This was the bit I was missing.

With use-package I would be able to declare which packages I needed, how they'd be installed and, most important of all, it could be set to handle the fact that the package wasn't even installed. If a package is requested and there is no local install use-package is smart enough to get the emacs package system to install it.

So, given that, all I need to do was create a startup file that would declare the packages I use and I'd have a setup that should, once I'd cloned .emacs.d, self-install.

Except... yeah, one more issue. use-package isn't part of GNU emacs yet so I'd need a method of getting it to auto-install so it could then handle everything else. As it was that was as easy as adding this to the start of my init.el.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Make sure the package system is up and running early on.
(require 'package)
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/"))
(add-to-list 'package-archives '("delpa" . "http://blog.davep.org/delpa/"))
(package-initialize)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Bootstrap `use-package'
(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))

With that in place I was able to nuke all my config on a machine, clone a fresh copy of .emacs.d (having now ceased tracking and storing the installed packages in that repo), run up emacs, wait a few moments and then find that everything was installed and ready to use.

Perfect!

My .emacs.d is now a lot smaller than it was before and, I think, even easier to maintain. Right now I think I'm very close to the ideal emacs config that I wanted to create when I did the complete rewrite a year ago.

Like many programmers, I have a couple of "Hello, World" projects that I've carried with me over the years. One is 5x5 (which has been used to get to grips with things as diverse as the Palm Pilot and GNU emacs). Another is Norton Guides database readers.

I've made Norton Guides tools that have allowed web servers to serve guides (w3ng), that have allowed you to convert guides to HTML (ng2html), that have let you read guides on OS/2 and GNU/Linux (eg) and also have let you read guides in Microsoft Windows (weg). It's a problem I know fairly well and one where I know the solution well enough so I can concentrate on learning the new language or environment.

Recently I wanted to get to grips with some "pure" ES6 coding while also getting to know node.js. A new version of the Norton Guide code, written for this environment, seemed like a good thing to do.

And so jsNG was born.

At its core is a library of code for opening and reading data from Norton Guides databases. While I doubt it's good ES6 code, or even good node.js code, it's been very useful in giving me a fun problem to solve and it'll carry on being something I'll tweak and tinker with by way of trying new things out.

On top of this I've built a handful of tools for working with Norton Guides databases. The most useful one at the moment (the others are more in the "test the library" than the "make something handy with the library" category) is ngserve. This is designed as a simple Norton Guides database HTTP server.

When run, you give it a list of guides to serve:

and it does the right thing. It has a small number of command line options that help configure what it does:

Possibly the most useful are the ones that let you change how it handles "higher" DOS characters and, if you don't like the default colours and stuff, the option that lets you point to your own style sheet (note for now you'll need to host the stylesheet somewhere else -- ngserve won't serve it for you; I'm aiming to change that in some way in the near future).

jsNG does have a fairly basic design compromise at its heart. In the very early version I started out using the async functions for opening and reading the guides. This got very tedious very quickly and I could see that it was going to make for a very messy library with a very messy interface. While it might not be in the spirit of node.js programming I decided to go with the sync version of the file IO functions and code up the core library based around this.

This approach also means that I took another leap that I never have done with Norton Guides before: rather than doing the traditional thing of keeping an open handle into them and reading direct from the file as you navigate the guide, I simply read it all into a buffer in one go and keep it in memory. This is a "guides are small, memory is cheap, things will go faster" approach.

It does mean that when you load up a load of guides into ngserve they're all sat in memory. The upside of this is that things should be a lot faster and the code is a lot easier to follow (I think). To put this in some perspective: I have a directory here that contains 110 Norton Guides files. They total 36M in size. If that seems like a lot of stuff to hold in memory... remind me how much is being used by your web browser so you can look at some hilarious kittens. ;)

Anyway, that's where I'm at with it right now. The code is mostly settled and mostly tidy. I need to write up some documentation for it (and so I need to take a look at good JavaScript documentation tools) and perhaps tinker with ngserve a little more. I'd also like to do a new version of ng2html with this -- a version that makes it far easier to control the style of the output. I'm also tempted to do a CLI-based reader in pure ES6; something similar to EG or WEG.

All in good time.

The other day I stumbled on a tip about Evernote. It was a snippet of information in a bigger post on the Evernote blog so I saved it to Evernote (obviously) to take a look at later.

The tip was that Ctrl-Cmd-B in Evernote (on the Mac) will format a body of text as source code. While I'm not in the habit of using Evernote to store code, not even snippets of code (that's something far better served by Gist), it seemed like something worth committing to memory.

This morning, while finally in front of a suitable machine, I took a look. Sure enough, there's the menu option.

Handy! So I created a test note with some code in it so that I could see how it formatted it. I was curious to see if it just did simple fixed text or if it offered options to highlight various languages (I didn't hold out much hope for the latter, but it was worth a look).

That's when it got odd.

After I created a note and went to format some code, the menu option disappeared!

From what I can tell, once it's gone, there's no way to get it back. At least, not until you actually close down Evernote and start it up again.

Frustratingly, none of this is the case in the Windows version. While the key combination is different there, the menu option is available and stays available.

The version of Evernote I have (on my iMac and my Macbook) is 6.9.2. The version number on Windows is 6.4.2 (after checking for updates -- I'm going to guess that the Mac and Windows version numbers don't match on purpose).

Goodness knows what's going on here. All I can imagine is that it has something to do with a thread I found on Evernote's support forum that suggests that code block formatting is some sort of test/beta feature and can only be enabled via a settings option that isn't available via the version downloaded from the Apple App Store.

So, at some point, I guess I'm going to have to uninstall Evernote from the Macbook and the iMac, reinstall from the Evernote website itself, and try this again. All of which seems a bit silly when the menu option is there and visible when I run Evernote up!

Edit to add: Sure enough, removing the App Store version and installing the version from Evernote's own website, and then going into preferences and enabling the option, sorted it. It's still really odd they'd promote the facility via the blog and not mention it, and also very odd that the option would always show until the first time you're in a position to use it, and then it'd disappear.