Posts tagged with "PyPI"

In the past few months, like a lot of folk in the Python world, I've been won over by uv. When it comes to managing my own projects my journey over the past few years as been pipenv, then rye, and then when rye was killed off I finally swapped to uv (later than I should have, I realised in hindsight). At each step I've found each tool cleaner, easier to work with and more feature-rich.

There's no doubt in my mind that uv has done the most work to make installing Python-based tools (or at least PyPI-based tools) as friction-free an experience as possible.

Now I've discovered uvx.sh. The thing with uv is the person installing your application first needs to get and install uv; this site removes that friction. Now if someone wants to install obs2nlm (for example) they should be able to just do:

curl -LsSf uvx.sh/obs2nlm/install.sh | sh

and away they go (assuming they have curl installed, which is generally going to be far more likely than having uv installed).

Of course, there are the usual caveats and concerns about the "just pipe this stuff via sh trust me bro" approach, but if you are comfortable with suggesting this kind of install method it looks worth adding this to an application's installation documentation.

I think I'm going to start mentioning it as an option.

After having knocked up obs2nlm the other day I realised there was another source of sources available to me that might be fun to throw into NotebookLM: one or more of the Norton Guide files in my collection!

And so, after a wee bit of hacking on a wet Sunday afternoon, ng2nlm was born.

In terms of what it does, it's pretty much the same as obs2nlm, only it uses ngdb to read the contents of a Norton Guide and turn it into a single Markdown file which can then be used as a source in NotebookLM.

So far it seems to be working a treat.

Here I grabbed the guide to assembly language Norton Guide, and the DOS interrupts Norton Guide, turned them into sources, created a notebook and let it do its thing.

I can't vouch for how valid the answer to my question is -- it's a long time since I wrote any 8086 assembler code and it's a moment since I last had to dig into DOS interrupts and the like -- but I like the result.

Probably the biggest issue I see at the moment is with the code examples: it seems to have a habit of adding some sort of citation marker that links to nowhere when it's emitting code. I think this is down to the instructions I've given it in the source I output, at least that's what it's suggesting when I ask it:

Me: What are all the numbers in square brackets for?

NotebookLM: The numbers in square brackets are passage indices used to cite which specific part of the sources supports each statement [Instruction]. Within the source files themselves, these markers identify distinct entries or sections within the Norton Guide databases.

Given I am trying to encourage it to cite its sources, I think this is where the confusion comes from. In "normal" paragraphs it does do a good job of citing its source and linking to it:

so presumably, when it "thinks" it's outputting code it doesn't do the markup to actually do the linking, and so things end up looking a little confused.

If I ask that citations aren't included in the code, this does seem to have the desired effect:

So, given this, perhaps I can use the --additional-instructions switch for the particular source to encourage it to not add citation links to code? Or of course I could tweak ng2nlm itself to include that instruction to start with. While it's a little specific to one use case, Norton Guide files do tend to be coding-related so it could make sense.

Anyway, in the very unlikely event that you have a need to turn one or more Norton Guide files into sources to throw at NotebookLM or similar tools: ng2nlm exists.

I'm sure I've mentioned a couple of times before that I've become quite the fan of Obsidian. For the past few years, at any given point, I've had a couple of vaults on the go. Generally I find such vaults a really useful place to record things I'd otherwise forget, and of course as a place to go back and look things up.

But... even then, it's easy enough to forget what you might have recorded and know that you can even go back and look things up. Also I tend to find that I can't quite figure out a good way of getting a good overview of what I've recorded, over time.

Meanwhile: I've been playing around with Google's NotebookLM as a tool to help research and understand various topics. After doing this with my recent winter break coding project (more on that in the future) I realised I really should get serious about taking this approach with my Obsidian Vaults.

I'm sure this is far from novel, I'm sure lots of people have done similar things already; in fact I'd quickly dabbled with the idea a few months ago, had a bit of a laugh at some of the things the "studio" made of a vault, and promptly forgot about it.

This time though I got to thinking that I should try and take it a little more seriously.

And so obs2nlm was born.

The idea is simple enough: enumerate all the Markdown files in the vault, wrap them in boundary markers, add some instructions to the start of the file to help NotebookLM "comprehend" the content better, throw in a table of contents to give clues to the structure of the vault, and see what happens when you use the resulting file as a source.

So far it's actually turning out to be really helpful. I've been using it to get summaries regarding my work over the past 18 months or so and it's helped me to consolidate my thoughts on all sorts of issues and subjects.

It's not perfect, however. I've had it "hallucinate" some stuff when making things in the studio (most notably in the slide deck facility); for me though I find this an acceptable use of an LLM. I know the subject it's talking to me about and I know when it's making stuff up. This, in turn, makes for a useful lesson in how and when to not trust the output of a tool like this.

Having tested it out with a currently-active vault, I'm now interested to find out what it'll make of some of the archived vaults I have. Back in 2024 I wrote a couple or so tools for making vaults from other things and so I have a vault of a journal I kept in Journey for a number of years, a vault of a journal I'd kept before then in Evernote, and I also have a vault of all the tweets I wrote before I abandoned Twitter. I also have a vault that was dedicated to recording the daily events and thoughts of my time working at Textualize. It's going to be fun seeing what NotebookLM makes of each of those; especially the last one.

Anyway, if Obsidian is your thing, and if you are dabbling with or fancy dabbling with NotebookLM, perhaps obs2nlm will be handy for you.

Back in the very early days of my involvement with Textualize, while looking for fun things to build to test out the framework and find problems with it, I created textual-astview. The idea was pretty simple: exercise Textual's Tree widget by using it to display the output of Python's abstract syntax tree module.

While the code still works, Textual has moved on a lot, as has my approach to building applications with Textual, and so I've been wanting to do a ground-up rewrite of it. At the same time I was also thinking that it might be interesting to build a tool that provides other ways of understanding how your Python source gets turned into runnable code; with this in mind I've ended up building a terminal-based application called DHV.

The idea is pretty straightforward: you type in, paste in, or load up, Python code, and you get a real-time display of what the resulting bytecode and abstract syntax tree would be.

If you've ever wondered what a particular bit of code looks like under the hood, or wondered if one particular approach to a problem is "more efficient"¹ than another, or just wondered to yourself if 1+1 ends up being a complex operation or simply gets turned into 2, this tool might be useful to experiment and see.

As of now DHV only works with Python 3.13. The main reason for this is that the Python dis module is a bit of a moving target and has had some noticeable interface changes over the past few versions. When I find some time I might work on making DHV a little more backward-compatible. But for now keep this in mind: when you're looking at the results for some code you're looking at what Python 3.13 (or later) would do, earlier Pythons may differ.

DHV is licensed GPL-3.0 and available via GitHub and also via PyPI. If you have an environment that has pipx installed you should be able to get up and going with:

pipx install dhv

If you're a fan of uv and friends you can install it like this:

uv tool install --python 3.13 dhv

Much like Norton Guide readers or the 5x5 puzzle, code that has fun with the Mandelbrot set is another one of my goto exercises. I've written versions in many languages, and messed with plots in some different environments, as varied as in VR on the web to wearable items.

Back in the early days of my involvement with Textualize I wrote a deliberately worst-approach version using that framework. The whole thing was about taking a really silly approach while also stress-testing Textual itself. It did the job.

Later on I did a second version that targets Textual. This time it did a better job and was the catalyst for building textual-canvas. This version was intended more to be a widget that happened to come with an example application, and while it was far more better than the on-purpose-terrible version mentioned above, I still wasn't 100% happy with the way it worked.

Recently I did some maintenance work on textual-canvas, cleaning up the repository and bringing it in line with how I like to maintain my Python projects these days, and this prompted me to look back at textual-mandelbrot and rework it too. Quickly I realised it wasn't really sensible to rewrite it in a way that it would be backward compatible (not that I think anyone has ever used the widget) and instead I decided to kick off a fresh stand-alone application.

Complexitty is the result.

Right now the application has all the same features as the mandelexp application that came with textual-mandelbrot, plus a couple more. Also it's built on top of the common core library I've been putting together for all my own terminal-based Python applications. As time goes on I'll add more features.

As with most of my recent TUI-based projects, the application is built with comprehensive help for commands and key bindings.

and there's also a command palette that helps you discover (and run) commands and their keyboard bindings.

Complexitty is licensed GPL-3.0 and available via GitHub and also via PyPI. If you have an environment that has pipx installed you should be able to get up and going with:

pipx install complexitty

It can also be installed with Homebrew by tapping davep/homebrew and then installing complexitty:

brew tap davep/homebrew
brew install complexitty

It pretty much all started with this:

 * Revision 1.1  1996/02/15 18:57:13  davep
 * Initial revision

That's from the rcs log for the source for w3ng, a tool I wrote so I could read Norton Guide files in my web browser, served by Apache, running on my GNU/Linux server in my office. The tool itself was written as a CGI tool (remember them?).

I believe I posted about this to comp.lang.clipper and pretty quickly some folk asked if it might be possible to do a version that would write the whole guide as a collection of HTML files for static hosting, rather than serving them from a cgi-bin utility. That seemed like a sensible idea and so:

 * Revision 1.1  1996/03/16 09:49:00  davep
 * Initial revision

ng2html was born.

Fast forward around a quarter of a century and I decided it would be fun to write a library for Python that reads Norton Guide files, and a tool called ng2web was the first test I wrote of it, designed as a more flexible replacement for ng2html. I've tweaked and tinkered with the tool since I first created it, but never actually "finished" it.

That's changed today. I've just released v1.0.0 of ng2web.

If turning one or more Norton Guide files into static websites seems like the sort of thing you want to be doing, take a look at the documentation.

ng2web is licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

pipx install ng2web

It can also be installed with Homebrew by tapping davep/homebrew and then installing ng2web:

brew tap davep/homebrew
brew install ng2web

It seems I really do still have this need to create new terminal-based projects at the moment. There's been Braindrop, then Peplum, then after that came Hike. While I'm still tweaking and adding to them, and also using them to refine a wee library I'm building up that forms the core of my latest apps, I felt I still had this one app that I needed to finally build.

Since the 1990s I've had this mild obsession with building tools for maintaining access to Norton Guide files. I've written readers for GNU/Linux (which also works on macOS too), OS/2, Windows, GNU Emacs, and also on the web (in multiple incarnations). Those builds have covered a few languages, including C, C++, Pascal, JavaScript and Emacs Lisp.

I'd never written a Python library or application for it though.

So when I first saw Textual mentioned in passing on Twitter a few years back, way back in the 0.1 days, I thought that could be the thing that would push me over the edge. In anticipation of that, back in 2021, I initially developed ngdb.py. This is a library that provides the core Norton Guide reading code for Python applications and could form the basis for other tools.

As a test for this I then wrote ng2web (which works, but I think still needs a bit of tidying up -- something I'm aiming to do in the next few weeks).

Meanwhile, the journey with Textual itself kicked off, happened, and came to an end; and yet somehow I'd never got round to building the thing I'd initially looked at Textual for: a terminal-based Norton Guide reader that looked nice and modern (by terminal standards). When I initially joined Textualize the owner had actually said they wanted me to build this as test of the framework, to essentially start out by employing me to create some Free Software that would help dogfood the library, but that seemed to get forgotten.

Fast forward to the start of this month and I finally felt it was time to tackle this. The result is AgiNG¹.

As of v0.1.0 it has most of the features you'd expect from a usable Norton Guide reader, including:

• An ability to add guide files to an in-application directory.
• The ability open and navigate a guide.
• Full see-also support, etc.
• Full translation of characters as were under MS-DOS into the terminal.
• The ability to copy entry text or source to the clipboard.
• The ability to copy save entry text or source to a file.
• Access to a guide's credits.

I still need to write some proper documentation for the application, but meanwhile all commands and key shortcuts can be discovered either via the help screen:

or by pulling up the command palette:

Hopefully the workings of the application will be fairly obvious to anyone who is familiar with Norton Guide files; if anything isn't making sense I'm more than happy to answer questions or take suggestions for improvements.

One wee feature I want to call out, that I felt was important to add, was a "classic view" facility. The thing with Norton Guide files is they were mostly created in the very late 1980s and early-to-mid 1990s. People would often get creative with the colouring within them, but in many cases the colouring assumed the default Norton Guide application. Its colours were white text on a blue background. So sometimes other colouring was done assuming that background.

You can see an example of this here, with an entry in a guide being viewed using the default textual-dark theme:

Notice the colouring in the syntax section. This is more obvious if the application is switched to one of the light themes:

With a nod to this issue in mind, I added the "classic view" for entries (which is a sticky setting -- turn it on and it stays on until you turn it off):

A little hard on the eyes, I think, but also filled with nostalgia!

Talking of themes, all the usual application themes are available, here's a wee selection:

AgiNG is licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

pipx install aging

It can also be installed with Homebrew by tapping davep/homebrew and then installing aging:

brew tap davep/homebrew
brew install aging

Expect to see more updates in the near future; as with other recent projects this is very much something I'm going to be dabbling with and improving as time goes on.

The run of writing new terminal-based tools that I want still keeps going. First there was Braindrop, then there was Peplum, and now, released today, there's Hike.

Hike is yet another terminal-based Markdown browser. While it's far from the first, and unlikely to be the last, it's mine and it looks and works exactly how I need. Perhaps it'll be your sort of thing too?

This initial release has a bunch of handy features, including things like:

• A command line where file names, URLs and commands can be entered.
• A persistent history for the command line.
• A local file browser.
• A simple bookmarking system.
• A browsing history.
• Commands for quickly loading and viewing files held on GitHub, GitLab, Codeberg and Bitbucket.

As there's a lot to discover in the application, I've tried to make the help screen as comprehensive as possible:

and there's also the command palette to help with discovering commands and the keys that are associated with them:

Once again, themes are supported so no matter your taste you should find something that's easy on your eyes:

Hike is licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

pipx install hike

It can also be installed with Homebrew by tapping davep/homebrew and then installing hike:

brew tap davep/homebrew
brew install hike

Expect to see more updates in the near future; this is very much an ongoing tinker project.

I seem to be back in the swing of writing handy (for me) little terminal-based applications again. Having not long since released Braindrop (which I'm still working on and tinkering with; it'll get more features in the near future, for sure), I had an idea for another tool I'd like to have: something for looking through, searching, and filtering Python PEPs.

As with anyone who is interested in what's happening with Python itself, I subscribe to the RSS feed of the latest Python PEPs, but I also wanted something that would let me look back at older ones in a way that worked "just so" ("just so" being "what feels right for me", of course). Having finished the main work on Braindrop I realised that the general layout of its UI would work here, as would the filtering and searching approach I used.

From this idea Peplum was born!

In this first release I've simply concentrated on all things to do with grabbing the list of PEPs and presenting them in a useful way; adding various forms of filtering them; adding the ability to search the metadata; that sort of thing. I aim to keep developing this out over the next few weeks and months, adding things like the ability to make notes, to locally view the text of a PEP, perhaps even to mark PEPs as unread and read, etc.

As I mentioned earlier, much of the design was driven by what I did with Braindrop, so once again I've tried my very best to make it keyboard-friendly and as much as possible keyboard-first. This sometimes means having to work against how Textual works, but generally that isn't too tricky to do. I'm once again making heavy use of the command palette and also ensuring that all commands that have corresponding keyboard bindings are documented in the help screen.

There's enough common code between Peplum and Braindrop, when it comes to this aspect of building a Textual application, that I'm minded to spin it out into a little library of its own. I'm going to sit on this code for a wee while and see how it develops, but I can see me taking this approach with future applications and doing this will stop the need to copy and paste.

It might also be useful to others building with Textual.

Also as with Braindrop, themes are a thing, and the theme setting is sticky so you can set it the once and stick with that you like. Here's some examples:

That's a small selection of the themes, with more to explore.

While working on this project I managed to find a couple more bugs in Textual, including a fun way to get transparent backgrounds to get out of sync and also a way to get an easy crash out of OptionList if it's set to width: auto.

What was even more fun was I sort of discovered a bug in the Python PEP API. Thanks to Hugo noticing my "huh, weird" post on Fosstodon, there's now an issue for it as well as a PR in the works. In retrospect I should have raised an issue myself; instead I fell into that "they obviously know what they're doing so it must be like this for a reason" trap.

Lesson relearned: it's always better to ask and get an answer, than to assume a thing is how it is for a reason you don't know; which I guess is a version of Linus' law really.

So that's v0.1.0 out in the wild. I'm pleased with how it's turned out and there's more to come. It's licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

pipx install peplum

It can also be installed with Homebrew by tapping davep/homebrew and then installing peplum:

brew tap davep/homebrew
brew install peplum

I'm going to be making good use of this and working on it more; I hope it's useful to someone else. :-)

A touch over a year ago I did the initial work on an application called Tinboard, a terminal-based client for the Pinboard bookmarking service. I had a lot of fun building it and it was an application that I used on a near-daily basis. However, around August last year I realised it was time for me to move on from Pinboard and try something new; based on various recommendations I settled on Raindrop.

As mentioned in the other blog post, Raindrop offered more or less everything I had with Pinboard and so the move was fairly straightforward. The one thing that was missing though was an application similar to Tinboard.

So, late on last year, with my winter break approaching, I decided to start from scratch and build a "Tinboard for Raindrop", which I'm calling Braindrop.

This was going to be a bit of an adventure too. Since being laid off from Textualize earlier in 2024 I'd not been following its development quite as closely as I used to, and had also run into some issues and bugs with it since that time; moreover, as well as various bugs appearing, some breaking changes had also been made. As such this was going to be a process where I'd wrap my head around what's happened with the framework over the prior six months or so.

Given all this, over the past couple of weeks I've been spending a few hours a day doing some for-pleasure coding and v0.1.0 of Braindrop is the result.

As much as possible I've tried to keep the look and feel similar to that of Tinboard, while also doing my best to avoid some of the "ah, I wish I hadn't done it this way" design decisions I'd made. As of the time of writing I'm very pleased with the result.

One thing I did want to do is ensure that the application was as keyboard-friendly as possible, while also still allowing use of the mouse. Textual can sometimes get that wrong and I ran into an example of this while trying to ensure that there's good in-application help. Somewhat recently Textual added a built-in help system which, sadly, can't easily be used by and navigated by someone using the keyboard. So instead I've recreated the help system I built into Tinboard, while adopting the documentation standard that Textual had settled on (which, coincidentally, was kind of similar to what I did in Tinboard to start with).

As with Tinboard, I've also made sure to make full use of the command palette, with every action that makes sense having a keyboard hotkey as well as a command in the palette. I also took things a little further and made sure that the hotkeys are shown in the command palette for easier discovery.

I've also made sure that Textual's new theme system is available for easy use; so out goes dark/light mode toggling and in comes a collection of different themes. Here's a wee selection as an example:

That's a small selection of the themes, with more to explore.

There's a few more things I want to do before I consider the application v1.0-ready, but it's already in use by me and working well. As I decide what else I want to add to it I'm building up a list of TODO items.

Given that my day job these days is quite varied, isn't quite so coding-intensive, and isn't always related to all things Python, it's actually been fun to sit down and hack up a pure Python application from scratch again. It's also helped me discover a couple or so fresh bugs in Textual (which I've reported, of course) and given me the opportunity to PR some trivial fixes as I've noticed typos and stuff as I go.

Anyway; that's v0.1.0 out in the wild. I'm pleased with how it's turned out and there's more to come. It's licensed GPL-3.0 and available via GitHub and also via PyPi. If you have an environment that has pipx installed you should be able to get up and going with:

pipx install braindrop

It can also be installed with Homebrew by tapping davep/homebrew and then installing braindrop:

brew tap davep/homebrew
brew install braindrop

I hope this is useful to someone else. :-)