Guile Steel: a proposal for a systems lisp

By Christine Lemmer-Webber on Sat 09 July 2022

Before we get into this kind of stream-of-consciousness outline, I'd like to note that very topically to this, over at the Spritely Institute (where I'm CTO, did I mention on here yet that I'm the CTO of a nonprofit to improve networked communication on the internet on this blog? because I don't think I did) we published a Scheme Primer, and the feedback to it has been just lovely. This post isn't a Spritely Institute thing (at least, not yet, though if its ideas manifested it could be possible we might use some of the tech), but since it's about Scheme, I thought I'd mention that.

This blogpost outlines something I've had kicking around in my head for a while: the desire for a modern "systems lisp", you know, kind of like Rust, except hopefully much better than Rust, and in Lisp. (And, if it turns out to be for not other reason, it might simply be better by being written in a Lisp.) But let's be clear: I haven't written anything, this blogpost is a ramble, it's just kind of a set of feelings about what I'd like, what I think is possible.

Let's open by saying that there's no real definition of what a "systems language" is... but more or less what people mean is, "something like C". In other words, what people nowadays consider a low-level language, even though C used to be considered a high level language. And what people really mean is: it's fast, it's statically typed, and it's really for the bit-fiddling types of speed demons out there.

Actually, let's put down a few asides for a moment. People have conflated two different benefits fo "statically typed" languages because they've mostly been seen together:

  • Static typing for ahead-of-time more-correct programs
  • Static typing for faster or leaner programs (which subdivides in terms of memory and CPU benefits, more or less)

In the recent FOSS & Crafts episode What is Lisp? we talk a bit about how the assumptions that dynamically typed languages are "slow" is really due to lack of hardware support, and that lisp machines actually had hardware support directly (tagged memory architecture and hardware garbage collection) and even wrote low-level parts of their systems like the "graphics drivers" directly in lisp, and it was plenty fast, and that it would even be possible to have co-processors on which dynamic code (not just lisp) ran at "native speed" (this is what the MacIvory did), but this is all somewhat of an aside because that's not the world we live in. So as much as I, Christine, would love to have tagged architecture (co-)processors, they probably won't happen, except there's some RISC-V tagged architecture things but I don't think they've gotten very far and they seem mostly motivated by a security model that doesn't make any sense to me. But I'd love to be wrong on this! I would like tagged RISC-V to succeed! But still, there's the problem of memory management, and I don't think anyone's been working on a hardware garbage collector or if that would really be a better thing anyway.

The fact is, there's been a reinforcing effect over the last several decades since the death of the lisp machine: CPUs are optimized for C, and C is optimized for CPUs, and both of them try to optimize for each other. So "systems programming" really means "something like C" because that's what our CPUs like because that's what our languages like and these are pretty much re-inforcing.

And besides, C is basically the lingua franca of programming languages, right? If you want to make something widely portable, you target the C ABI, because pretty much all programming languages have some sort of C FFI toolkit thing or just make C bindings, and everyone is happy. Except, oh wait, C doesn't actually have an ABI! Well, okay, I guess not, but it doesn't matter because the C ABI triples, that's what the world works with.

Well also, you gotta target the web, right? And actually the story there is a bit nicer because WebAssembly is actually kinda awesome, and the hope and dream is that all programming languages in some way or another target WebAssembly, and then "you gotta write your thing in Javascript because it's the language of the web!!!" is no longer a thing I have to hear anymore. (Yes, all my friends who work on Javascript, I appreciate you for making it the one programming language which has mostly gotten better over time... hopefully it stays that way, and best of luck.) But the point is, any interesting programming language these days should be targeting Webassembly, and hopefully not just via Emscripten, but hopefully via actually targeting Webassembly directly.

So okay, we have at least two targets for our "system language": C, or something that is C-compatible, and Webassembly. And static type analysis in terms of preventing errors, that's also a useful thing, I won't deny it. (I think the division of "statically typed" and "dynamically typed" languages is probably more of a false one than we tend to think, but that's a future blogpost, to be written.) And these days, it's also how you get speed while also being maximally bit-twiddly fast, because that's how our machines (including the abstract one in Webassembly) are designed. So okay, grumbling about conflating two things aside, let's run with that.

So anyway, I promised to write about this "Guile Steel" thing I've been musing about, and we've gotten this far in the article, and I haven't yet. So, this is, more than a concrete proposal, a call to arms to implement just such a systems language for Guile. I might make a prototype at some point, but you, dear reader, are free to take the idea of "Guile Steel" and run with it. In fact, please do.

So anyway. First, about the name. It's probably pretty obvious based on the name that I'm suggesting this be a language for Guile Scheme. And "Guile" as a name itself is both a continuation of the kind of playfully mischevious names in the Scheme family and its predecessors, but also a pun on co-founder of the Scheme language, Guy L. Steele. So "Guile Steele" kinda brings that pun home, and "Steel" sounds low-level, close to the metal.

But also, Guile has a lovely compiler tower. It would be nice to put some more lovely things on it! Why not a systems language?

There's some precedent here. The lovely Scheme 48's lowest levels of code (including its garbage collector) are written in an interesting language called PreScheme (more on PreScheme), which is something that's kind of like Scheme, but not really. It doesn't do automatic garbage collection itself, and I think Rust has shown that this area could be improved for a more modern PreScheme system. But you can hack on it at the REPL, and then it can compile to C, and it also has an implementation on Common Lisp, so you can bootstrap it a few different ways. PreScheme uses a Hindley-Milner type system; I suspect we can do even better with a propagator approach but that's untested. Anyway, starting by porting PreScheme from Scheme48 to Guile directly would be a good way to get going.

Guile also has some pretty good reasons to want something like this. For one thing, if you're a Guile person, then by gosh you're probably a Guix person. And Rust, it's real popular these days, and for good reasons, we're all better of with less memory vulnerabilities in our lives, but you know... it's kind of a pain, packaging wise, I hear? Actually I've never tried packaging anything in Rust but Efraim certainly has and when your presentation starts with the slide "Packaging Rust crates in GNU Guix: How hard could it possibly be?" I guess the answer is going to be that it's a bit of a headache. So maybe it's not the end of the world, but I think it might be nice if on that ground we had our own alternative, but that's just a minor thing.

And I don't think there's anything wrong with Rust, but I'd love to see... can we do better? I feel like it could be hackable, accessible, and it also could, probably, be a lot of fun? That's a good reason, I know I'd like something like this myself, I'd like to play with it, I'd like to be able to use it.

But maybe also... well, let's not beat around the bush, a whole lot of Guile is written in C, and our dear wonderful Andy Wingo has done a lot of lovely things to make us less dependent on C, some half-straps and some baseline compilers and just rewriting a lot of stuff in Scheme and so on and so forth but it would be nice if we had something we could officially rally around as "hey this is the thing we're going to start rewriting things in", because you know, C really is kind of a hard world to trust, and I'd like the programming language environment I rely on to not be so heavily built on it.

And at this point in the article, I have to say that Xerz! pointed out that there is a thing called Carp which is indeed a lisp that compiles to C and you know what, I'm pretty embarassed for having not paid attention to it... I certainly saw it linked at one point but didn't pay enough attention, and... maybe it needs a closer look. Heck, it's written in Haskell, which is a pretty cool choice.

But hey, the Guile community still deserves a thing of its own, right? What do we have that compiler tower for if we're not going to add some cool things to it? And... gosh, I'd really like to get Guile in the browser, and there are some various paths, and Wingo gave a fun presentation on compiling to Webassembly last year, but wouldn't it be nice if just our whole language stack was written in something designed to compile to either something C-like or... something?

I might do some weekend fiddling towards this direction, but sadly this can't be my main project. As a call to arms, maybe it inspires someone to take it up as theirs though. I will say that if you work on it, I promise to spend some time using whatever you build and trying it out and sending patches. So that's it, that's my stream-of-consciousness post on Guile Steel: currently an idea... maybe eventually a reality?