Lucy McPhail's blog

Things I know about beets

2026-07-22T00:00:00+00:00

god damn lucy I need you to just list everything you know about beet</code>. You keep saying things that make me go “whaaa what I wish I knew that before”
Louis Pilfold</a></cite> </blockquote>
Sure thing! I’ve been using beets for a few years now to organise my music library, and I’ve picked up a few tricks along the way. In short, beets is a media library management system, which helps you catalogue your music collection and improve its metadata by drawing from databases like MusicBrainz and Discogs. I’m not going to cover the basics here though; for that, I’d recommend going straight to their getting started</a> guide. These are a handful of slightly less common features that I find myself using :)
First, a note on enabling plugins, since I’m about to talk about a few of them. In order to enable a plugin, you can add that plugin’s name to the plugins</code> option in your config.yaml</code> file. You can find the location of your config file either by running beet config -p</code>, or by checking the docs</a>. Open your config file in your text editor, and add a section that looks like this:
plugins:
 - fish
 - replaygain
 - fetchart
 - convert
 - missing
 - duplicates
 # whatever else you like</code></pre>fish completions</a></h1>
I use the fish shell, so I really like that beets comes with a plugin to easily get high-quality shell completions for fish. Enable the fish</code> plugin, then run beet fish</code>, and beets will create a file called ~/.config/fish/completions/beet.fish</code>. You should now be able to tab-complete beets commands!
ReplayGain</a></h1>
ReplayGain is a standard for measuring and normalising the perceived loudness of audio tracks. If your music player supports ReplayGain, it can read the relative loudness of each track or album, and adjust the playback gain so that they all sound equally loud.

 
 what did you mean by "track or album"?
</blockquote>
When you’re playing back your music, you can choose whether you’d like every track to sound equally loud, or for gain to be applied equally to every track in an album. With the latter option, differences in perceived loudness between tracks in an album are preserved, but whole albums will be normalised relative to each other. Beets’ ReplayGain analysis will compute both, so you can usually choose which you’d like in your music player’s settings. Personally, I use album mode.
To generate ReplayGain metadata from beets, install beets[replaygain]</code> and enable the replaygain</code> plugin, and optionally choose an alternate backend. I use the ffmpeg</code> backend, since it’s pretty quick and supports lots of audio formats. If you’d like to do that, make sure you have ffmpeg</code> installed, and then add this to your configuration:
replaygain:
 backend: ffmpeg</code></pre>
By default, the ReplayGain plugin will automatically analyse new tracks as you import them (which is a little slow, don’t worry if it looks like it’s hanging for a bit), but if you’d like to analyse your existing library, run beet replaygain</code> and it’ll go through everything that isn’t already analysed.
For more details, I refer you to the docs</a>.
fetching album art</a></h1>
I like seeing album art when I’m scrolling through my library! Sometimes you’ll get album art when you download an album, like when you get one from Bandcamp, but sometimes you’ll have to get it yourself. Mercifully, beets can automate the process of trawling the internet looking for album art. Install beets[fetchart]</code> and enable the fetchart</code> plugin, and then whenever you import an album, it’ll automatically search the internet for album art, and save it in the album’s folder as cover.jpg</code>. Like the ReplayGain plugin, you can also add album art to your existing library by running beet fetchart</code>, which will look for album art for any albums in your library that don’t already have it.
Some music players, such as iPods, don’t check for a cover.jpg</code> file, so you’ll have to embed the album art into each audio file. Beets also has a plugin for this: install beets[embedart]</code> and enable the embedart</code> plugin (you’ll also need fetchart</code> to get the images in the first place), and then beets will automatically embed images whenever fetchart</code> gets a new cover.
Once again, for more details, see the docs pages for fetchart</a> and embedart</a>.
converting between formats</a></h1>
The beets convert</code> plugin is a real multi-tool. You can use it to re-encode files in your library, replacing the old ones, or to automatically encode files as you import them. My favourite use, though, is to transcode lossless files to opus in order to save space on music players, while keeping the original lossless files in my library. It’s also useful if you’re using a music player which has limited codec support, like the stock firmware on an iPod, and you need to transcode your library in order to play it at all.
In order to do that, enable the convert</code> plugin, and then add this to your configuration:
convert:
 format: opus
 never_convert_lossy_files: yes</code></pre>
Then, run beet convert --dest /mnt/media/your_music_player</code>, changing the destination path to the mount point of your own music player. With the settings above, this will transcode any lossless files to opus, but leave lossy files untouched since converting from one lossy codec to another will always decrease quality without necessarily saving much space. If you’re using an iPod and need to convert everything to AAC, for example, then omit the never_convert_lossy_files</code> option from your configuration and change the format to aac</code>.
You can also include a query, in order to transcode only part of your library, but I’ll go into queries later on. Here’s the doc page for convert</a>.
finding missing or duplicate tracks</a></h1>
Nice quick one: do you ever accidentally end up with tracks missing from your albums, or import something twice and end up with duplicates? Enable the missing</code> and duplicates</code> plugins, then you can run beet missing</code> to see which of your albums are missing tracks, or beet duplicates</code> to see duplicate tracks. Here are the docs for missing</a> and duplicates</a>.
querying and list formats</a></h1>
Pretty much any time you run a command in beets that acts on your library, you have the option to supply a query to limit which part of your library the command will act on. The simplest type of query is a keyword search: if I run beet ls gold</code>, I see
David Bowie - Best of Bowie - Golden Years
Fleetwood Mac - Rumours - Gold Dust Woman
MARINA - FROOT - Gold
Panic! at the Disco - Death of a Bachelor - Golden Days
Periphery - Periphery III: Select Difficulty - Marigold
Lena Raine - Celeste Original Soundtrack - Golden
Sting - Ten Summoner’s Tales - Fields of Gold
...</code></pre>
Beets can also search query by specific fields: beet ls artist:green"</code> will turn up Grant Green and Ted Greene, but not Eddi Reader’s Green Grow the Rashes O. I’ll not go through every query feature – the query docs</a> do a good job of that already – but I do have some neat uses for it.
Let’s say we want to figure out the total length of an album. We can start off by getting all the tracks in the album we’re interested in, using the :=</code> syntax to search for exact matches, and quote marks to look for a whole phrase rather than a keyword:
$ beet ls album:="The Rainbow Goblins"
高中正義 - The Rainbow Goblins - Prologue
高中正義 - The Rainbow Goblins - Once Upon a Song
高中正義 - The Rainbow Goblins - Seven Goblins
高中正義 - The Rainbow Goblins - The Sunset Valley
高中正義 - The Rainbow Goblins - The Moon Rose
高中正義 - The Rainbow Goblins - Soon
高中正義 - The Rainbow Goblins - Magical Night Light/Rainbow Paradise
高中正義 - The Rainbow Goblins - Thunderstorm
高中正義 - The Rainbow Goblins - Rising Arch
高中正義 - The Rainbow Goblins - Just Chuckle
高中正義 - The Rainbow Goblins - Rainbow Was Reborn
高中正義 - The Rainbow Goblins - Plumed Bird
高中正義 - The Rainbow Goblins - You Can Never Come To This Place</code></pre>
So now we know which songs are in the album, but we’d rather see how long they all are rather than their names. We can use beets’ format syntax</a> to get there:
$ beet ls album:="The Rainbow Goblins" -f '$length'
3:15
5:10
3:12
[...]</code></pre>
We’ll want those in plain seconds:
$ beet ls album:="The Rainbow Goblins" -f '$length' \
 | xargs -I '{}' date -u -d "1 jan 1970 0:{}" +%s
195
310
192
[...]</code></pre>
 
 sorry, what did you just do with date??
</blockquote>
Ah, that! It takes a duration like 3:15, and turns it into “1 jan 1970 0:3:15” – 3 minutes and 15 seconds after the epoch – and then outputs that timestamp as seconds since the epoch. It’s a hack, and it breaks if you have tracks that are over an hour long, but it’s good enough here.
Now, we can sum up the seconds with awk</code>:
$ beet ls album:="The Rainbow Goblins" -f '$length' \
 | xargs -I '{}' date -u -d "1 jan 1970 0:{}" +%s \
 | awk '{ total += $1 } END { printf "%s\n", total }'
3797</code></pre>
And then use date</code> again to turn it into hours, minutes, and seconds:
$ beet ls album:="The Rainbow Goblins" -f '$length' \
 | xargs -I '{}' date -u -d "1 jan 1970 0:{}" +%s \
 | awk '{ total += $1 } END { printf "%s\n", total }' \
 | xargs -I '{}' date -u -d "@{}" +%H:%M:%S
01:03:17</code></pre>
And we’re done! It’s worth having a look at the list of format fields that beets can output, since there are heaps of them, and I’ve found it really useful on a few occasions now. I wanted to transcode all of my ALAC files to FLAC a while back, and finding the ALAC files was as easy as beet ls format:flac -f '$path'</code>. Then I can pass that whole list to beet format</code> with the --keep-new</code> flag, and that was that!
I really like beets, and it’s helped me keep my music library far more organised than it ever would have been otherwise, and enabled some really esoteric workflows with how scriptable it is. Please let me know if you’ve got any weird beets tricks of your own, and hopefully I can make a part 2!

Using Caddy as a link shortener 2026-01-17T00:00:00+00:00 A few days ago, I was chatting to Cassidy</a> after finding out she plays go, and she casually dropped a link to her profile at cass.run/ogs</a>. I link people to my own OGS account from time to time, but that link looks like online-go.com/user/view/1687228</code>, which is a little tough to remember. Usually I just tell people my username and let them look it up, but that’s a bit more hassle for them. seven digits? that doesn't look so bad </blockquote> Well I guess I could remember my account number, but I use it infrequently enough that it’s easy to forget in between the times that I need it. And besides, I have plenty more URLs that would be useful to have on hand, but that are pretty tough to remember. All this is to say, I really like Cassidy’s idea of a personal URL shortener. Hers lives, aptly, at cass.run/shortener</a>, but it’s deployed as a Netlify application. I already have all of my domains pointing to my reverse proxy, Caddy, so I’d rather just use that if I can. Turns out it’s actually pretty easy! Just add this to your Caddyfile: rosy.systems { map {path} {redirect-uri} { /ogs https://online-go.com/user/view/1687228 /path https://whatever.other.url.you.like default https://lucymcphail.com } redir {redirect-uri} }</code></pre> Replace the domain at the top with your own, and make sure to leave in the default</code> mapping if you want it to gracefully handle paths that you haven’t added a redirect for. It works fine if you don’t, but you’ll get an ugly default error page if you try and hit /wibble</code> or something else you haven’t defined. From there, just add any extra shortened URLs you’d like! Happy shortening :) Outlook keeps logging me out 2025-10-06T00:00:00+00:00 I started at a new job a few months ago, and as it so often does, it came along with a new Microsoft email address. This time though, I’m using a Linux machine, and I don’t have permission to use a mail client other than Outlook. That leaves the web version of Outlook as my only choice. Predictably, it hasn’t been smooth sailing. So here’s what happens when I try and open Outlook in Firefox: I log in with my password and TOTP code, and… it lets me in. Seems fine so far. </blockquote> Sure! Until the next morning, when I have to log in again. I sign in, exactly the same way, and this is what happens: So I let it sign me out, close the tab, and try again. Same thing. And better still, since TOTP codes are, well, one-time codes; I’m essentially rate limited to two attempts a minute while I wait for the codes to roll over. Usually it works eventually, but I’ve spent 15 minutes some mornings trying to get into my inbox. In case anyone reading this is in the same boat, here’s what works for me: in Firefox settings, then Privacy & Security, under Cookies and Site Data, click on “Manage Data…”. From there, select microsoft.com and outlook.com, and click Remove Selected for both. For now, at least, that lets me log back into Outlook first try. Fixing my scrollbars 2024-11-04T00:00:00+00:00 I don’t know when it started happening, or why. Maybe it’s because people are getting so used to touch screens on smartphones, maybe it’s because designers want to make their interfaces as minimal as possible; I can’t say. But I like a good, old-fashioned scrollbar, and unfortunately they seem to be going the way of the dodo. I have a few problems with the new style of overlay scrollbar, the kind that disappear after a while if you leave them alone. For one, the scrollbar gives me two useful pieces of information if I’m reading an article: first, the size of the scrollbar lets me see roughly how long the article is; and second, I can see how far through the page I am. It makes me a little sad that some websites feel the need to add a fancy progress bar at the top of the page to let me see where I am – this is a solved problem! This is what scrollbars are for! The other issue I have with overlay scrollbars is, y’know, using them for scrolling! If I want to scroll with a conventional scroll bar, I can see the whole scrollbar and I can either click on the thumb (the little rectangle that moves) and drag it, or click somewhere along the track (the big rectangle the thumb moves along) to jump there. With overlay scrollbars, I have to wiggle my mouse over the rough area I think the scrollbar is to summon it, and then drag the absolutely teensy thumb, and there isn’t a track to click on anymore, so that’s not an option. wow, I didn't know anyone cared so much about scrollbars... </blockquote> Okay, perhaps I’m nitpicking, but I stand by the fact that having nice chunky, persistent scrollbars makes it easier to use my computer. Fortunately, it’s pretty easy to get them back! firefox</a></h1> Most of the time if I’m on my computer, I’m either in emacs or firefox, and for the former I tend not to use scollbars anyway. After all, it’s not often I need to know how far down I am in a code file, and even when I do, the percentage progress indicator at the bottom is good enough for me. Suffice it to say, I really want my scrollbars in firefox to be good. Luckily, there are a few hidden settings we can use to get them that way. First, there’s an option in the general settings menu to always show scrollbars. Enabling this will give you a fairly sensible scrollbar that stays on the side of the window where you left it. This is a pretty good start, but there are some hidden settings in about:config</code> that are worth playing with as well. Here are the ones I change: layout.css.scrollbar-width-thin.disabled = true</code> stops web pages from making your scrollbar thin in CSS</li> widget.gtk.overlay-scrollbars.enabled = false</code> seems to stop a few scrollbars in weird places from appearing? I’m not quite sure what this one does honestly, but it might be handy</li> widget.non-native-theme.gtk.scrollbar.allow-buttons = true</code> puts the buttons back at the top and bottom of the scrollbar if you’re on linux and your GTK theme has them; I’ll get to that in a moment!</li> widget.non-native-theme.scrollbar.style = 4</code> chooses between a few shapes for your scrollbar. I like 4 the best, since it gives you a lovely, chunky, deliciously clickable, rectangular scrollbar.</li> widget.non-native-theme.scrollbar.size.override = 20</code> changes the width of the scrollbars, so you can make the click target as big as you like</li> </ul> Here’s what my scrollbars look like after all of that: GTK</a></h1> I said I’d mention my GTK theme since Adwaita, the default in most places, doesn’t have arrows on scrollbars. I’m enjoying the chicago95</a> theme at the minute, which gives me my scrollbar buttons back in firefox, and as an added bonus it makes all my other GTK applications look delightfully charming: That’s all for now, but if any of you know any other scrollbar related tricks, I’d love to hear about them <3 RC 2024: halfway already!? 2024-09-26T00:00:00+00:00 That’s all, see you next week! :) </blockquote> — An overly optimistic Lucy, about four weeks ago So, uh, it’s been a little while. The last few weeks have been a bit tough on me for reasons outside of RC or programming in general, and to be honest I’ve barely had the energy to work on my projects, let alone write here. I don’t think the weekly update format I’ve been aiming for so far works very well for me: it starts feeling like more of a chore, something I have to do rather than something I do because I want to. I do want to keep writing here, and more regularly than I have been, but I think in order to do that I need to give myself a little more flexibility. With any luck, it’ll help me be a little gentler on myself and I won’t stress myself out of writing anything. If anyone has any advice for dealing with this sort of writer’s block, I’d love to hear it. Another thing that’s been getting in the way of me writing these posts is feeling like I don’t have a lot to write about. I’ve been making slow progress over the last few weeks, but a lot of it doesn’t feel interesting enough to write about. I’ve learned lots of little things along the way, but I don’t have progress that feels like it’s ‘worth’ showing off here, if that makes any sense? I recently read Nicole’s blog post on content-driven development</a>, where she talks about projects getting stuck because of overwhelming task lists – a sentiment to which I can deeply relate – and reframing them in terms of sharable milestones. She says that being able to split projects into deliverable pieces can help with project direction (if it isn’t worth explaining to other people, maybe it doesn’t need to be there yet?) as well as generally staying motivated. It’s a great post, and I’m looking forward to trying some of her ideas in my own projects. As for what I’m actually planning on doing, I definitely want to branch out into a few more projects so that if I’m losing momentum on one, I can pick up steam again working on another. I’m going to keep working on my chess engine; in the spirit of content-driven development, I’d like to finish the move generation so I can actually watch it play a game of chess (if not a good one quite yet), and potentially put it up on my server for people to play against. Another thing I’ve been interested in for a little while is building a debugger, and there’s a group at RC already working through a course on that so I’d quite like to give that a go. It also seems like a good project to test Zig</a> with, since Anya</a> has been speaking very highly of it lately and says it seems like exactly the kind of language I like. I think I’d like to have one more, slightly more artsy and creative project going as well, for when I’m in that kind of mood, but I’m not sure exactly what I’d like that to be yet. I have a few ideas, but I think I’ll save that for next week. One more thing I’d like to do: there are a handful of new faces in the category theory group so we’re doing a bit of a reset, and since Milewski’s book has proved to be a bit confusing and lacking in detail, we’re going to try and pick a different book to go through. So, along with the others, I’m going to sample a few different textbooks and see if any stand out as being particularly approachable without mathematical background. I’m not sure if I’ll follow through on this yet, but learning about homotopy type theory and univalent foundations is something I’ve wanted to do for a while, so I might pick up the HoTT book again and see if more of it sticks than last time I tried. halfway there</a></h1> It’s an exciting time at RC at the moment, since this week marks the end of the Summer 2 batch and the beginning of Fall 2. It’s been bittersweet: I’m really enjoying meeting all the new people and I’m so excited to see what they get up to, but it does also mean saying goodbye to a few friends. I’m so grateful for the time I got to spend with all the Summer 2 folks, and I’ve met a few people I really want to stay in touch with as they go off and do other things. Being in the second half of my batch also means I need to start thinking about my own life after RC, and in particular, it’s time to start looking for a job. I’m not sure if I’ll post anything about that here, since I don’t expect my job hunt to be particularly interesting, but it does seem like that’ll be a notable part of the next few weeks. For now though, I’m still very happy to be at RC and I’m feeling optimistic about what I can do with the next six weeks. I’m not sure quite when the next post here is going to be, but with any luck there’ll be something interesting in there! And thanks so much to all my friends at RC who’ve helped me make cool stuff over the last couple of weeks, I love you all <3 RC 2024 week 2: on... beer and categories? 2024-08-27T00:00:00+00:00 Soooo. I was going to write this post a couple of days ago on Sunday, but some kind soul decided to douse me with beer from the passenger seat of his mate’s car while I was walking back to my flat. Cleaning the errant beverage from my clothes and myself, it turns out, is not an activity particularly conducive to getting in the mood to write. Some rest and a few episodes of Deep Space Nine later, I’m feeling a bit better, so here we are. It’s been a bit of a low energy week for me in general, so I’ve been more focused on doing some reading and getting some smaller, less challenging work done. For example, I’ve tried to make this site a little more readable and added a form for folks to subscribe by email if they prefer. I haven’t done so much with my chess engine this week, but the FEN parsing and a basic UCI implementation are done so I just need to crack on with the move generation. I think next week I’d like to try and pair on it with some more people, if for no other reason than having some company while I work on the slightly less interesting bits. *hugs* </blockquote> Thank you, Lily. category theory things</a></h1> In other news, I’ve had some interesting chats with the category theory group this week, especially about some of the exercises in Milewski’s book. In particular, this one from the first chapter: When is a directed graph a category? </blockquote> This question confused a few people in the group, and in my opinion, rightfully so! Saying a directed graph is never a category seems to me a perfectly reasonable answer, in the same way a banana is never a grapefruit. A category simply has more structure than a directed graph – namely categories have composition – and there is no canonical way to add this structure to a digraph. To use a programming analogy, I might ask when a block of memory is a struct. Right; a block of memory might be an instance of a struct, but you couldn't possibly know unless you knew what the struct was! If someone just handed you some raw bytes, then sure, you could turn it into a struct, but you'd have to choose which structure to impose on it. You can't just say yes or no without making some decisions first. </blockquote> Exactly! So let’s weaken the question a little bit: can we turn any directed graph into a category? Sure! Take the graph, throw it in the bin, and replace it with the empty category :D </blockquote> I– yeah, I guess I can’t argue with that. I’m pretty sure the idea behind that question was to highlight the differences between a category and a directed graph: in a category, if we have any two composable morphisms, their composition must also exist; and every object must have a morphism, from it back to itself, that satisfies the identity laws. But instead, for our group, it ended up serving as a great example of why mathematicians are so insistent on being precise in wording our statements. My thanks to the category theory group for asking me some hard questions and making me think. wrapping up</a></h1> Since I’m writing this post a little late, ‘next week’ is actually this week for me. Still, I have more than half the week left, so it still feels reasonable to talk about my plans. I’m very much enjoying my role in clarifying Milewski’s material for the rest of the reading group, so I’m tempted to put a bit of time into making some materials of my own to present. I’d love to see my chess bot actually playing a game of chess, so I’m going to try and push through writing the move generation code. And finally, Nicholas convinced me to give a talk with him on the basics of solving cryptic crosswords, so I’ll be writing up some notes for that. If it goes well, I might even turn it into a blog post! That’s all, see you next week! :) RC 2024 week 1: finding community 2024-08-18T00:00:00+00:00 So, that’s my first week at the Recurse Center over and done with! And my word, what a week it’s been. I tried to spend as much time as possible going to anything on the calendar that looked interesting, which means I didn’t have much time to write code, but I met a lot of people and got to see some awesome projects so I don’t regret my decision. First of all, everything I’ve heard about the culture at RC is completely true so far. As a member of several marginalised groups, I often find myself feeling like an outsider in tech spaces. Being in a field heavily dominated by cis white men, it’s all too easy to feel somewhat othered, and that my voice isn’t being heard. At Recurse, though, I’ve felt absolutely none of that. It takes a lot of work to create and maintain such a diverse and welcoming culture, and the faculty as well as the rest of the RC community continuously work to do so. More than any other technical space I’ve been part of, I feel like I truly belong at RC. what has lucy been up to?</a></h1> So. Much. I decided to keep a written log of what I’ve been doing with my time this week, and I’m incredibly glad because there’s no way I would have been able to remember everything myself. My batch started with the introductory meet and greets. These were structured a bit like a speed dating event, so I met a lot of people in a very short time. It was an absolutely amazing start: only a couple of hours into the first day and already I’d met a whole bunch of my batchmates and heard about all the interesting things they were working on. Then us remote folks got a chance to explore the software side of RC. As well as the usual suspects like a chat system (in this case, Zulip) and calendar, there’s also a whole virtual Recurse Center! It’s a 2D version of the physical hub that runs in a browser, where you can see which zoom rooms people are in because their little avatar is standing there. It’s a little thing but I think having a shared space really adds to the remote experience. There’s also a coffee chat bot that randomly assigns you another person to chat to on days of your choosing. I’ve had coffee chats nearly every day so far, and they’ve been a great way to get to know some of my batchmates. Perhaps my favourite introductory event was the pairing workshop: you and another Recurser get a simple brief – for us, it was writing an implementation of Conway’s game of life – and then you have an hour to implement it together. It was a great way to start pairing with people in a low-pressure environment, and I definitely feel more comfortable asking people to pair on my actual projects now. Most of the events at RC are organised not by the faculty, though, but by other Recursers. There are a few long-running events that have carried over from before my batch, since the Summer 2 batch is still here for another six weeks, and I decided to check out quite a few of them. Among my favourites were the groups on operating systems and systems programming, practical language design, and the Nix language and tooling. There were also a few of us who were interested in category theory, so this week marked the beginning of our category theory study group! Unfortunately I doubt I’ll have time to keep attending all of these events if I want to start making progress on my own projects, but I’m glad I checked out so many just to see what was happening. For now, I think I’m going to stick with the category theory and systems programming meet-ups. And then there were the presentations! First, there are non-programming talks every week on Wednesdays. I’m not sure what I was expecting to hear about but it definitely wasn’t tiramisu and moss. It turns out tiramisu is a lot easier to make than I had assumed, so I’ll have to give that a go now (upon telling my partner and my parents this, they all suddenly seemed to be quite hungry – something tells me I’m not going to get a choice in the matter but you won’t hear me complaining about it heheh). Then there are the programming talks that take place weekly on Thursdays. To name a few of my favourite topics: developing a command line poker game; automatically taking screenshots from a reMarkable tablet and embedding them in a note in Obsidian by calling one function; and connecting a Raspberry Pi to the internet by playing a T9-encoded WiFi password with DTMF tones into the handset of a landline phone. All of these, and the many others I didn’t mention here, were fantastic and I’m very glad I got to see them. woah dude that's... a lot </blockquote> Yep! It’s been a very social week for me, and I’ve enjoyed meeting everyone and going to all the events, but next week I definitely want to get some more technical work done and pair with more people. I did get a little bit of programming done this week though: I’ve been learning the basics of Go and getting the toolchain all set up, and started laying the groundwork for my chess engine. I’ll write a separate, more technical post about what exactly goes into that, but in short, I have to encode the rules of chess so I can tell which moves are legal from a given position, and teach the bot to speak to other chess programs so I can play against it. so what’s the plan?</a></h1> As I mentioned earlier, I’d like to dial down the social stuff from this week and start making more significant progress on my technical projects. With that in mind, here are my goals for the next week or so: for the chess engine, I’m hoping to have at least the legal move generation working by the end of next week. I’d also like to make sure I have comprehensive tests and some benchmarks set up: getting move generation right is extremely important, and since chess engines play with clocks, ‘thinking’ time is limited so I want to get benchmarking working from the beginning. So far, though, I’m really enjoying the support for testing and benchmarking built into Go.</li> I’ve been fascinated by eBPF since I discovered it a couple of years ago, and I’d love to try and write some actual tools with it. The application I had in mind back then was a sampling profiler, which still sounds fun, but I’m open to being distracted here.</li> I’ll carry on with the category theory group; so far I’ve read through the first three chapters of Bartosz Milewski’s Category Theory for Programmers. I have a formal background in category theory from my maths degree, but I’m curious to see the more advanced applications for working programmers as opposed to, say, homotopy theorists.</li> </ul> Don't worry, homotopy theory, we still love you <3 </blockquote> That’s about all I wanted to say about this week, so I suppose there’s nothing else to do but get on with it! So with that, dear reader, I bid you adieu; see you again this time next week :) I’m going to the Recurse Center! 2024-08-12T00:00:00+00:00 Today is my first day at the Recurse Center</a>! For anyone who hasn’t heard of it before, it’s a 12 week programming retreat: essentially, I have the next three months to work on whatever projects I like, and try and learn as much as possible, alongside a group of extremely cool people with exactly the same aims. I’m incredibly grateful to be able to attend, and I’m so excited to get started. I first heard about RC through Julia Evans</a>, and later again from Nicole Tietz-Sokolskaya</a>, both of whom wrote regular updates during their time at RC. Reading about their experiences was by far my biggest inspiration to apply, and now that I’m finished with my maths degree, I’m finally starting a batch myself! It only seems right, then, that I should pick up the torch and write about my own experiences here. Perhaps, someday, someone will read this and it’ll help them decide whether applying to Recurse is the right choice for them :) So... what are you actually going to do? </blockquote> Well, one post I found really helpful deciding how to set my goals is this one</a> from RC alum Ray Fong on maximising your time at RC, so I’m going to steal a couple of their ideas: first, keeping my plans flexible so that I can change what I’m doing if something piques my interest; and second, having distinct technical and non-technical goals. First, the technical: My friend Cosmo, author of one of the world’s strongest chess engines</a>, has been trying to convince me to write a chess bot of my own for years now. Now that I have 12 weeks set aside for this sort of project, I think it’s finally time to take them up on it!</li> I’d love to know more about programming language design, so I’m planning to go through Crafting Interpreters</a> and potentially make some more interesting languages. The end goal here is to play with interesting type systems (linear types, session types, dependent types, and so on) so I’d like to make myself a framework to play with them in.</li> A few other little things I’d like to tackle if I have time: I’ve always been fascinated by modern multi-process OSs, specifically how their schedulers work, so I’d like to try and make my own toy one. I’ve also been meaning to play about with theorem provers like Lean</a> for a while, and see if I can figure out how they tick.</li> </ul> As for my non-technical goals: Above all, I’d love to make some new friends. RC prides itself on being a diverse and inclusive community, so it seems like the perfect place to meet some wholesome, friendly programmers who share some of my interests.</li> I’d also like to take some time to look after myself. Doing a maths degree has left me feeling very tired and burned out, and I’m hoping RC will be the kick I need to get excited about making things again. On top of that, I’d like to try and improve my diet and get some more sunshine and exercise while I’m in my batch.</li> </ul> So that’s my plan! I’m a big fan of working transparently, so I’ll make sure to post links to anything I’m working on in case anyone wants to look at the code. Plus, I’d like to write weekly updates on this blog; partly to keep myself accountable, but also so anyone can follow along. I’m so excited to get started, and to all the Recursers reading this, I can’t wait to meet you <3 Deploying this blog on NixOS 2024-08-08T00:00:00+00:00 For a while now I’ve been wanting to move this website from a managed service (in my case, sourcehut pages) to a server I run myself, and recently I got around to actually doing it. That said though, I really like the user experience these managed services provide: just git push</code> your site and it gets automagically built and deployed for you. I think I’ve managed to recreate that experience in NixOS, and I wanted to write down what I did for anyone else who’s curious. generating the site</a></h1> My blog is built with Zola</a>, a lightweight static site generator which comes as a single binary with no dependencies. As a result, it’s already very easy to build reproducibly; no faffing with npm install</code>, just one zola build</code> and we’re off to the races. Still, there are a few reasons I like to build it with a Nix flake. Flakes are an experimental* feature of Nix which provide a standard format for declaring packages whose dependencies are pinned in a lock file. This is great for my site, since it lets me build it on any machine I like knowing that the zola</code> binary is identical on each. It’s also handy for deployments, since the output of a flake gets added to the Nix store (/nix/store/</code>) each time it gets built, which means I can old builds are always around if I want to roll back. *well, 'experimental'</summary> Flakes have actually been around since 2019 and are mostly stable at this point, but there’s some controversy around them that hasn’t fully resolved itself yet. In short, the feature was implemented after the original RFC</a> was closed, which made some people feel like flakes were implemented without enough input from the community. In the years since, a large ecosystem has sprung up around flakes, and I think it’s fair to say most of the community would like them to be stable. As of last year, there’s a new RFC</a> hoping to stabilize things, but for now flakes are still behind a feature flag. </details> Here’s the flake.nix</code> for this site: { description = "A flake to generate my personal website"; inputs = { nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable"; flake-utils.url = "github:numtide/flake-utils"; }; outputs = { self, nixpkgs, flake-utils }: flake-utils.lib.eachDefaultSystem (system: let pkgs = nixpkgs.legacyPackages.${system}; in { packages.site = pkgs.stdenv.mkDerivation rec { name = "lucymcphail.com"; src = ./.; nativeBuildInputs = [ pkgs.zola ]; buildPhase = "zola build"; installPhase = "cp -r public $out"; }; defaultPackage = self.packages.${system}.site; devShell = pkgs.mkShell { packages = [ pkgs.zola ]; }; } ); }</code></pre> Running nix build</code> will build the site with Zola and then copy the output to $out</code>, which Nix sets to somewhere inside /nix/store</code>, and creates a symlink to it called result</code> for convenience. The other advantage of using flakes this way is that they can include local development shells, so I can preview my site locally without having to install Zola for my whole system. The devShell</code> output lets you run nix shell</code> to get a shell with Zola ready to use. Better still, I use nix-direnv</a> to automatically load the development shell when I’m in the flake directory, so I can run zola serve</code> without having to think about it. configuring caddy</a></h1> Now that we have the site content, it’s time to get it up on the server! I decided to try out caddy, and I’m very happy with it; the configuration language is simple, the documentation is good, and most impressive of all, it can automatically obtain and update your TLS certificates! NixOS provides a services.caddy</code> module so it’s easy to set up. We’ll want to create a www</code> group so that other users can write to /var/www</code>, and make sure /var/www</code> exists with systemd.tmpfiles</code> (I’m not sure why, but this seems to be a common way to create folders with Nix, even if they’re not temporary). Then, we can enable caddy, tell Nix to add the automatically generated caddy</code> user to the www</code> group, and serve the site. We’ll add this to our server’s /etc/nixos/configuration.nix</code> (or wherever you keep your config): users.groups.www = {}; systemd.tmpfiles.rules = [ "d /var/www 0775 caddy www" ]; services.caddy = { enable = true; group = "www"; extraConfig = '' lucymcphail.com { encode zstd gzip root * /var/www/lucymcphail.com file_server } ''; };</code></pre> One nixos-rebuild switch</code> later and once caddy obtains some TLS certificates for us, we have a website! Of course, we could just update the contents of the site with scp</code> and call it a day, but I think it’s much more fun to do it with git :) running a git server</a></h1> Using git over ssh</code> is actually shockingly easy: just run git push <user>@<host>:/path/to/git/repo</code> and as long as the repository already exists on that machine, it should just work! There are a few things we can do to make it a bit nicer, though: users.users.git = { isSystemUser = true; group = "www"; home = "/srv/git"; createHome = true; shell = "${pkgs.git}/bin/git-shell"; openssh.authorizedKeys.keys = [ # your ssh key here ]; };</code></pre> So now we have a dedicated git user, complete with its own home directory for us to store our repos in. It’s also using the git shell as its login shell, which allows restricted access to git commands over ssh (see man git-shell</code>). Now, if we create our repos in /srv/git</code>, we can access them with git@<host>:<repo-name></code>. It’s a good idea to use bare git repositories here, in case we have multiple users contributing. A bare git repo is the same as a regular one, but without a working tree; that is to say, a bare repo just contains the contents of the .git</code> folder in a non-bare repo. This is perfect for a server, since otherwise we could end up with the working tree going out of sync and causing problems when we try to push. bonus: git frontend</a></h2> As an aside, we can also set up a git frontend so we can view our repositories from the web. I like legit</a> for this, since it’s tiny and lightweight compared to larger software development services like gogs or gitea, and I don’t care about being able to edit files or create accounts. Once again, we can set it up using Nix: services.legit = { enable = true; user = "git"; settings.repo.mainBranch = [ "trunk" "main" ]; settings.repo.scanPath = "/srv/git"; };</code></pre> We’ll run it under the git user too, so that it can read our repos, and tell it where to read them from. I’m one of those strange people who likes using trunk as a branch name, so I’ll also tell legit to recognise that as a default branch. By default, it’ll serve on 127.0.0.1:5555. building the site with a hook</a></h2> Back to the website! We’ll create a new bare repo with git init --bare /srv/git/lucymcphail.com</code>, and now we can create a hook to build the site whenever new changes are pushed. Git hooks live inside the hooks</code> folder (.git/hooks</code> in a non-bare repo), and usually there are a few example scripts in there in a new repo. Of particular interest to us is the post-receive</code> hook, which runs whenever someone pushes to the server. Git hooks are just shell scripts, so we can build the site from there: #!/usr/bin/env bash set -eu SITE="lucymcphail.com" while read oldrev newrev ref do if [[ $ref =~ .*/trunk$ ]]; then echo "Trunk received. Deploying..." mkdir -p /tmp/$SITE git --work-tree=/tmp/$SITE --git-dir=$HOME/$SITE checkout -f trunk pushd /tmp/$SITE nix build /tmp/$SITE rm -rf /var/www/$SITE/* cp -r --no-preserve=mode result/* /var/www/$SITE popd else echo "Ref $ref successfully received. Doing nothing: only trunk may be deployed on this server." fi done</code></pre> The post-receive</code> hook gets called with a list of refs being passed through stdin, so we can read through them all to check if trunk has been pushed, since we don’t want to deploy the site from a feature branch. If there is an update to trunk, we’ll check out the repository into /tmp/lucymcphail.com</code> and build it with Nix. Once Nix is done, it’ll store the site somewhere in /nix/store</code> and symlink /var/www/lucymcphail.com</code> to it. Finally, caddy will start serving the updated version of the site. The whole deployment happens in just a few seconds. Here’s what that looks like: $ git push buttercup Pushing to buttercup:lucymcphail.com Writing objects: 100% (6/6), 553 bytes | 553.00 KiB/s, done. Total 6 (delta 5), reused 0 (delta 0), pack-reused 0 (from 0) remote: trunk received. deploying... remote: Already on 'trunk' remote: /tmp/lucymcphail.com ~/lucymcphail.com remote: this derivation will be built: remote: /nix/store/xgij95fj95j4v9s1z2pfixajnx1ihrch-lucymcphail.com.drv remote: building '/nix/store/xgij95fj95j4v9s1z2pfixajnx1ihrch-lucymcphail.com.drv'... remote: ~/lucymcphail.com To buttercup:lucymcphail.com 0dffb7f..cdde960 trunk -> trunk</code></pre> So that’s how this site runs! Updating it is really smooth now, just like github pages et al. but with significantly faster build times. I really like this setup so far, and everything being managed by Nix should mean it’s not too hard to maintain and I can roll it back if something breaks. Emacs packages I like 2024-07-29T00:00:00+00:00 During the time I’ve been using Emacs, a handful of packages have stood out to me as being particularly useful, reliable, and generally enjoyable to use. A few of them in particular, I like enough to write about :) Modus themes</a></h1> One of the first things I see when I start a new Emacs session is my colour theme, so we may as well start there. My theme of choice has been one of Protesilaos Stavrou’s Modus themes</a> for almost as long as I’ve been using Emacs, and I’ve yet to find anything else that shares Prot’s attention to detail. The themes are designed with accessible readability in mind, so the contrast between the background and foreground colours is always high. On top of that, variants of both the dark and light themes are included which are optimised for users with red-green or blue-yellow colour deficiency. Colours alone don’t make an Emacs theme though, and it’s everything else about the Modus themes that really shines. The highlighting is well thought out, with colours being used only where they’re useful; the coverage of third-party packages is great; and there are a staggering number of customisation options. They’re an unapologetically usable set of themes, and I love them for it. Vertico/orderless/marginalia</a></h1> Lots of things in Emacs use the built-in completing-read</code> function to get user input while completing from a list of candidates. The default behaviour is to provide an input field with bash-like tab-completions, but sometimes it’s nicer to show the list of candidates on screen and filter it with a fuzzy finder. Daniel Mendler’s vertico</a>, for ‘VERTical Interactive COmpletion’, adds the first of these features, orderless and adds the second. For example, enabling vertico-mode</code> will turn this: Find file: ~/src/some-project/█</code></pre> into this: */7 Find file: ~/src/some-project/█ Cargo.lock Cargo.toml LICENSE README.md src/ target/ .gitignore</code></pre> where the list of files gets filtered down as you type. The benefit of vertico’s approach, using the completing-read</code> mechanism instead of redefining each function, is that is plays nicely with other parts of Emacs. For example, Omar Antolín Camarena’s orderless</a> defines a completion style which tells Emacs how to filter and sort candidates, and vertico will use that ordering with no extra configuration. Orderless is a great companion to vertico, as it sorts completions based on how recently they were last used, and filters them fuzzily based on the input string (for example typing car lo</code> will match Cargo.lock</code>), making vertico act like the fuzzy finders in more modern editors. Marginalia</a>, another of Daniel’s packages, improves vertico once again by showing extra details beside each candidate where possible: for example, showing file details in the file picker or function docstrings in M-x</code>. For very minimal setup, these three packages vastly improve the completion experience in Emacs. project.el</code> and Magit</a></h1> Emacs’ built-in project</code> package is a neat little feature that works right out of the box, but it’s far too useful not to mention here. It adds a handful of helpful functions for dealing with projects (which it automatically detects by looking for version controlled directories), of which I have a few favourites: C-x p p</code>: switch to a different project</li> C-x p f</code>: find files in project</li> C-x p c</code>: run compile</code> in the project root</li> C-x p g</code>: project-wide grep</li> C-x p !</code>: execute a shell command from the project root</li> </ul> In particular, C-x p f</code> in combination with vertico is a great fuzzy file finder, similar to Quick Open in Visual Studio Code or fzf</code>/telescope in (neo)vim. Complementing project</code>, Magit</a> is one of the best Git frontends I’ve ever used anywhere, let alone for Emacs. As a testament to just how good it is, its developer, Jonas Bernoulli, started a Kickstarter campaign in 2017 asking for CHF 55 thousand to develop Magit full time for an entire year. He raised almost CHF 73 thousand. As a result, Magit is perhaps the most feature-complete Git frontend beside the Git command-line. It aims to look similar to the output of Git itself, but everything in a Magit buffer is actionable. You can stage and unstage files by pressing s</code> and u</code> in the status window, or rebase onto a particular commit by pressing r</code> with the cursor on that commit in the log. Even if you don’t use Emacs for anything else, I still think Magit is worth a try on its own. Eglot</a></h1> LSP is a relatively new addition to my setup compared to everything else here, since I was content with using keyword completion and tags for the features I needed – that is, until I tried rust-analyzer</code>, and now writing Rust doesn’t feel the same without it. In Emacs, lsp-mode</code></a> and eglot</code></a> seem to be the two most popular LSP implementations, the latter of which is included in Emacs 29 onwards. While lsp-mode</code> seems to be the more feature-rich of the two, eglot</code> takes an approach similar to Vertico of being minimal and building on core Emacs features wherever possible. As well as, in my opinion, feeling more responsive than lsp-mode</code>, it also means any configuration you may already have for flymake</code> or completion-at-point-functions</code> will work identically with LSP features. In particular, Corfu</a> (once again from Daniel Mendler) works perfectly with eglot</code> out of the box. I really don’t have much else to say about eglot</code>; it just seems to work whenever needed, and stay out of the way when it isn’t. Using an iPod after 20 years 2024-06-14T00:00:00+00:00 I use an iPod mini, made in 2004, to listen to music every day. Why, when streaming makes music so much more convenient and accessible, would I choose to ignore two decades of technological progress? Well, there are two aspects to music streaming that I don’t find particularly satisfying. The first is that I don’t agree with the business model of streaming. Letting users listen to an enormous catalogue for the price of a single record a month simply isn’t sustainable; it’s no coincidence that all of the major streaming services have either the backing of large companies like Google and Apple, or a large amount of VC funding, as in the case of Spotify. It also means musicians see much less money from streaming than they would from selling digital downloads or physical media, and as a musician myself I’m much more sympathetic to their cause than Spotify’s. I also don’t love the way music is presented by streaming services. Often on Spotify I would find myself taking the path of least resistance and putting on a large playlist on shuffle, or listening to their automatically generated mixes, without much thought about what I was listening to. Maintaining my own collection of music and actively choosing what to play feels much more intentional, and I like that now my recommendations come from my friends instead of from an algorigthm. This still doesn’t paint the complete picture, though: why carry around a 20 year old music player instead of just using the phone I already have to take with me anyway? Truthfully, part of the reason is nostalgia. I grew up listening to music on an iPod nano, and something about scrolling around with the clickwheel brings me a little bit of joy every time I use it. But it’s also nice having a device designed just for listening to music, instead of using my phone. It never pauses my music to let me know I have a message, and its always-on monochrome LCD lets me see the current track in direct sunlight. With that said, there are still a few things we can do to bring an old iPod up to speed with modern music players. Part of the reason I chose the iPod mini specifically is that it’s relatively easy to get into compared to some newer models, and iFixit have a great guide</a> for taking them apart. Both the battery and the 4 GB hard drive that came with the iPod are starting to show signs of age, but both can be easily replaced. I swapped out the battery for a new one, and replaced the hard drive with an SD to compact flash adapter from iFlash</a> and a spacious 256 GB SD card. With these modifications, this iPod is totally usable as my main music player. Managing a music library with beets</a></h1> Luckily, despite the rise of streaming, there are still plenty of ways to get digital music. Lots of indie musicians sell digital downloads on Bandcamp</a>, and some even let you buy CDs, vinyl, or cassettes if that’s more your style. For music more than a few years old, ripping second hand CDs is a great way to get high quality files and support local record shops in the process. The difficult part is organising files from all these disparate sources, and making sure they have consistent metadata. Fortunately for us, the team behind beets</a> already thought of that! Beets is a handy little program that takes care of renaming files and sorting them into a sensible structure, as well as tagging each file with the correct metadata, and much more if you use the included plugins. I won’t explain how it works here, because the docs already do a great job of that, but I highly recommend it if you have a large library of music library you need to keep track of. Switching to rockbox</a></h1> While I do enjoy the default firmware that ships with the iPod mini, partly for nostalgic reasons and partly because I find it elegantly simple, it does have a few notable shortcomings. For me, the most important of them are: not being able to play music in formats like vorbis, opus, and FLAC;</li> requiring the use of iTunes to transfer music; and</li> not understanding the ‘album artist’ id3 tag, which means albums featuring guest artists sometimes get split up (for example, ‘Panic! at the Disco feat. LOLO’ is considered its own artist, so Miss Jackson doesn’t get included in Too Weird to Live, Too Rare to Die if you search by artist).</li> </ul> Rockbox is a community-made firmware for a collection of different music players, including my iPod mini. While it does have its own quirks, and feels a lot less simple than the stock firmware, it does fix each of my issues. It has robust support for modern codecs and gapless playback, it lets you add music just by copying it over USB, and it handles album artists properly. Installing rockbox was easy using their automated installer</a>, and getting it set up was painless. I still don’t like the way it looks and feels as much as the original firmware, and I haven’t tested it rigorously, but it seems like the battery isn’t lasting quite as long. Nevertheless, I do like rockbox enough to keep using it daily. Aside: syncthing</a></h1> Unfortunately, using a 20 year old music player with third-party firmware isn’t the most stable thing. A few times now, I’ve pulled my iPod out of my bag only to find that the battery is completely empty despite it being plugged in the night before, and naturally these days it’s hard to find people with a 30-pin cable to borrow. So, then, it’s useful to have a backup music player with me. For reasons beyond my understanding, my phone, despite being 20 years newer and several times more expensive, has neither expandable storage nor a headphone jack. Nonetheless, it will suffice as a replacement music player. Since I was already using it, syncthing seemed like a natural choice to keep my library up to date with the copy on my computer. Syncthing is a wonderful piece of software: simply set it up on each of the devices you want to sync between, and it’ll diligently keep whichever folders you choose in sync across each device, over LAN or the Internet. The upshot is all I need to do to add new music is run beet import</code> on my laptop to add it to my library, and syncthing will automatically copy it to my phone. Ultimately, I still much prefer the experience of using a dedicated device for playing music, but it’s good to know I have a backup for whenever it decides not to work. Recreating Apple’s sync</a></h1> One thing I think Apple absolutely nailed about the iPod was the experience of syncing with iTunes. Being able to plug an iPod into your computer and have it automatically sync any new music is really convenient. Using rockbox, adding new music is just a matter of copying files, but wouldn’t it be nice if we could have the computer check which files are missing and just copy those? Well, that’s exactly what rsync</code> does! I have this script to copy my library onto my iPod’s default mountpoint on my system: #!/usr/bin/env bash rsync -a --progress --size-only \ "$HOME/Music/" \ "/media/lucy/LUCY'S IPOD/Music"</code></pre> The -a</code> flag runs rsync</code> in archive mode, so it will copy the entire directory tree, and I use --size-only</code> here to skip files that match in size since the iPod’s USB interface is pretty slow, so not having to do a checksum to see if each file has changed is a significant speedup. I really like having a dedicated device for playing music. I can see myself switching to another device at some point, in particular the tangara</a> looks really promising, but I don’t think I’ll be going back to streaming as my main form of music consumption any time soon.