{.align-center}

Try running "telnet postcard.sfconservancy.org 2333" in your terminal... you won't regret it, I promise! You should get to see a fun "season's greetings postcard" animated live in your terminal. My friends at Software Freedom Conservancy are running a supporters drive and I was happy to do this to help (if you were a supporter in the past, you should be getting a matching postcard in the mail!)

As you probably guessed, this is built on top of the same tech stack as Terminal Phase. Speaking of, at the time of writing we're getting quite close to that... only $40/month to go! Maybe we'll make it by the end of the year?

In the meanwhile, if you want to show your support for Conservancy and convince your friends to join, there are some matching banners you can put up on your website! And if you want to run the animated postcard yourself or generate any of its associated images, well... it's free software!

{.align-center}

Yeah you read (and saw, via the above gif) that right! A space shooter! That runs in your terminal! Pew pew pew!

Well it's most of one, anyway. It's a prototype that I built as a test program for Spritely Goblins.

I've satisfied the technical needs I had in building the program; I might still finish it as a game, and it's close enough where making a satisfying game rather than just a short demo is super feasible, but I've decided to see whether or not there's actually enough interest in that at all by leaving that as a milestone on my Patreon. (We're actually getting quite close to meeting it... would be cool if it happened!)

But what am I, a person who is mostly known for work on a federated social web protocol, doing making a game demo, especially for a singleplayer game? Was it just for fun? It turns out it has more to do with my long term plans for the federated social web than it may appear.

And while it would be cool to get something out there that I would be proud of for its entertainment value, in the meanwhile the most interesting aspects of this demo to me are actually the technical ones. I thought I'd walk through what those are in this post, because in a sense it's a preview of some of the stuff ahead in Spritely. (Now that I've written most of this post, I have to add the forewarning that this blogpost wanders a lot, but I hope all the paths it goes down are sufficiently interesting.)

Racket and game development

Before we get to the Spritely stuff, I want to talk about why I think Racket is an underappreciated environment for making games. Well, in a certain sense Racket been advertised for its association with game-making, but in some more obscure ways. Let's review those:

• Racket has historically been built as an educational teaching-programming environment, for two audiences: middle schoolers, and college freshmen. Both of them to some degree involve using the big-bang "toy" game engine. It is, by default, functional in its design, though you can mix it with imperative constructs. And yet maybe to some that makes it sound like it would be more complicated, but it's very easy to pick up. (DrRacket, Racket's bundled editor, also makes this a lot easier.) If middle schoolers can learn it, so can you.
• Along those lines, there's a whole book called Realm of Racket that's all about learning to program by making games. It's pretty good!
• Game studio Naughty Dog has used Racket to build custom in-house tools for their games.
• Pioneering game developer John Carmack built prototypes for the Oculus Rift using Racket. (It didn't become production code, though John's assessments of Racket's strengths appear to align with my own largely; I was really pleased to see him repost on twitter my blogpost about Racket being an acceptable Python. He did give a caveat, however.)

So, maybe you've already heard about Racket used in a game development context, but despite that I think most people don't really know how to start using Racket as a game development environment. It doesn't help that the toy game engine, big-bang, is slowed down by a lot of dynamic safety checks that are there to help newcomers learn about common errors.

It turns out there are a bunch of really delightful game development tools for Racket, but they aren't very well advertised and don't really have a nice comprehensive tutorial that explains how to get started with them. (I've written up one hastily for a friend; maybe I should polish it up and release it as its own document.) Most of these are written by Racket developer Jay McCarthy, whom I consider to be one of those kind of "mad genius hacker" types. They're fairly lego-like, so here's a portrait of what I consider to be the "Jay McCarthy game development stack":

• Lux, a functional game engine loop. Really this is basically "big-bang for grown-ups"; the idea is very similar but the implementation is much faster. (It has some weird naming conventions in it though; turns out there's a funny reason for that...)
• By default Lux ships with a minimal rendering engine based on the Racket drawing toolkit. Combined with (either of) Racket's functional picture combinator libraries (pict or the 2htdp/image library), this can be used to make game prototypes very quickly, but they're still likely to be quite slow. Fortunately, Lux has a clever design by which you can swap out different rendering engines (and input mechanisms) which can compose with it.
• One of these engines is mode-lambda which has the hilarious tagline of "the best 2d graphics of the 90s, today!" If you're making 2d games, this library is very fast. What's interesting about this library is that it's more or less designed off of the ideas from the Super Nintendo graphics engine (including support for Mode-7 style graphics, the graphic hack that powered games like the Super Nintendo version of Mario Kart). Jay talks about this in his "Get Bonus!" talk from a few years ago. (As a side note, for a while I was confused about Get Bonus's relationship to the rest of the Jay McCarthy stack; at 2018's RacketCon I queried Jay about this and he explained to me that that's more or less the experimental testing grounds for the rest of his libraries, and when the ideas congeal they get factored out. That makes a lot of sense!)
• Another rendering engine (and useful library in general) is Jay's raart library. This is a functional picture library like pict, but instead of building up raster/vector graphic images, you're building up ascii art. (It turns out that ascii art is a topic of interest for me so I really like raart. Unsurprisingly, this is what's powering Terminal Phase's graphics.)

As I said before, the main problem with these is knowing how to get started. While the reference documentation for each library is quite good, none of them really have a tutorial that show off the core ideas. Fortunately each project does ship with examples in its git repository; I recommend looking at each one. What I did was simply split my editor and type in each example line by line so I could think about what it was doing. But yeah, we really could use a real tutorial for this stuff.

Okay, so 2d graphical and terminal programs are both covered. What about 3d? Well, there's really two options so far:

• There's a very promising library called Pict3d that does functional 3d combinators. The library is so very cool and I highly, highly recommend you watch the Pict3d RacketCon talk which is hands down one of the coolest videos I've ever seen. If you use DrRacket, you can compose together shapes and not only will they display at the REPL, you can rotate around and view them from different angles interactively. Unfortunately it hasn't seen much love the last few years and it also mostly only provides tools out of the box for very basic geometric primitives. Most people developing 3d games would want to import their meshes and also their skeletal animations and etc etc and Pict3d doesn't really provide any of that yet. I don't think it even has support for textures. It could maybe have those things, but probably needs development help. At the moment it's really optimized for building something with very abstract geometry; a 3d space shooter that resembles the original Star Fox game could be a good fit. (Interestingly my understanding is that Pict3d doesn't really store things as meshes; I think it may store shapes in their abstract math'y form and raytrace on the GPU? I could be wrong about this.)
• There's also an OpenGL library for Racket but it's very, very low level. (There's also this other OpenGL library but I haven't really looked at it.) However raw OpenGL is so extremely low level that you'll need to build a lot of abstractions on top of it to make it usable for anything.

So that covers the game loop, input, display.

That leaves us with audio and game behavior. I'll admit that I haven't researched audio options sufficiently; rsound seems like a good fit though. (I really need a proper Guix package of this for it to work on my machine for FFI-finding-the-library reasons...)

As for game behavior, that kind of depends on what your game is. When I wrote Racktris I really only had a couple of pieces of state to manage (the grid, where the falling piece was, etc) and it was fairly easy to do it using very basic functional programming constructs (really you're just creating a new version of these objects every time that a tick or input happens). As soon as I tried moving to a game with many independently operating objects doing their own thing, that approach fell apart.

The next thing I tried using was Racket's classes and object system. That was... okay, but it also felt like I was losing out on a lot. Previously I had the pleasant experience that "Woo, yeah! It's all functional!" A functional game engine is nice because returning to a prior snapshot in time is always possible (you can trivially make a game engine where you can rewind time, for instance; great for debugging) and it's much easier to poke and prod at a structure because you aren't changing it, you just are getting it back. Now I lost that feature.

In that conversation I had with Jay at RacketCon 2018, he suggested that I look at his DOS library for game behavior. I won't as quickly recommend this one; it's good for some game designs but the library (particularly the DOS/Win part) is fairly opinionated against objects ("processes") communicating with each other on the same tick; the idea is that each processes only can read what each other wrote on the previous tick. The goal here as I understand it is to eliminate an entire class of bugs and race conditions, but I quickly found that trying to work around the restrictions lead me to creating terrible hacks that were themselves very buggy.

This became really clear to me when I tried to implement a very simple wild west "quick draw" game a-la the Kirby quick draw and Samurai Kirby type games. (All this happened months back, when I was doing the anniversary animation I made earlier this year.) These are very simple games where two players wait for "Draw!" to appear on the screen before they press the button to "fire their gun". Fire first, you win. Fire second, you lose. Fire too early, you lose. Both fire at the same time, you draw. This is a very simple game, but trying to build it on top of DOS/Win (or my DOS/Hurd variant) was extremely hard to do while splitting the judge and players into separate objects. I ended up writing very contorted code that ultimately did communicate on the same tick, but via a layered approach that ended up taking me an hour to track down all the bugs in. I can't imagine scaling it up further.

But DOS had some good ideas, and I got to thinking about how to extend the system to allow for immediate calls, what would it look like? That's when I hit a series of epiphanies which resulted in a big rewrite of the Spritely Goblins codebase (which turned out to make it more useful for programming game behavior in a way that even fits very nicely into the Lux game loop). But I suppose I should really explain the what and why of Spritely Goblins, and how it fits into the larger goals of Spritely.

Terminal Phase and Spritely Goblins and stuff

Spritely Goblins is part of the larger Spritely project. Given Spritely's ambitious goal of "leveling up" the fediverse by extending it into the realm of rich and secure virtual worlds, we have to support distributed programming in a way that assumes a mutually suspicious network. (To get in the right mindset for this, maybe both watch my keynote and Mark Miller's keynote from the ActivityPub conference.) We really want to bake that in at the foundation of our design to build this right.

Thus Spritely Goblins is an asynchronous actor-ish distributed programming system on top of Racket. Kind of like Erlang, but with a focus on object capability security. Most of the good ideas have been taken from the E programming language ("the most interesting programming language you've never heard of"). The only programming environments I would consider viable to build Spritely on top of are ones that have been heavily informed by E, the best other candidate being the stuff Agoric is building on top of Javascript, such as their SwingSet architecture and Jessie (big surprise, since the folks behind E are mostly the folks behind Agoric), or some other more obscure language environments like Monte or, yes, Goblins. (Though, currently despite hanging out in Racket-land, which drinks deeply into the dream of everyone building their own languages, Goblins is just a library. If you want to run code you don't trust though, you'll have to wait until I release Spritely Dungeon, which will be a secure module / language restriction system for Racket. All in due time.)

Spritely Goblins already has some interesting properties:

• All objects/actors are actually just procedures, waiting to be invoked! All "state" is merely the lexical scope of the enclosed procedure. Upon being invoked, a procedure can both return a value to its invoker (or in asynchronous programming, that fulfills the promise it is listening to) as well as specify what the next version of itself should be (ie, what procedure should be called the next time it handles a message).
• Objects can only invoke other objects they have a reference to. This, surprisingly, is a sufficient security model as the foundation for everything we need (well, plus sealers/unsealers but I won't get into those here). This is the core observation from Jonathan Rees's A Security Kernel Based on the Lambda Calculus; object capability security is really just everyday programming via argument passing, which pretty much all programmers know how to do. (This applies to modules too, but more on that in a future post.)
• In most cases, objects live in a "vat". This strange term from the object capability literature really means an event loop. Objects/actors can send messages to other objects in other vats; for the most part it doesn't matter where (on what machine, in what OS process, etc) other objects are when it comes to asynchronous message passing.
• When asynchronous message passing, information is eventually resolved via promises. (Initially I had promises hidden behind coroutines in the core of the system, but it turns out that opens you to re-entrancy attacks if you aren't very careful. That may come back eventually, but with great care.)
• While any object can communicate with any other object on any vat via message passing, objects on the same vat can do something that objects on separate vats can't: they can perform immediate calls (ie, something that looks like normal straight-ahead programming code, no coroutines required: you invoke the other object like a procedure, and it returns with a value). It turns out this is needed if you want to implement many interesting transactional things like financial instruments built on top of pure object capabilities. This also is nice for something like a game like Terminal Phase, where we really aren't doing anything asynchronous, are running on a fixed frame rate, and want to be deterministic. But a user should remember that (for important reasons I won't get into in this post) that immediate calls are strictly less universal than asynchronous message passing, since those can only be done between objects in the same vat. It's pleasant that Goblins can support both methods of development, including in an intermixed environment.
• There is actually a lower level of abstraction than a vat, it turns out! This is something that is different than both E and Agoric's SwingSet I think and maybe even mildly novel; all the core operations (receiving a message, spawning an actor, etc) to operate on the actormap datastructure are exposed to the user. Furthermore, all of these operations are transactional! When using the lower-level actormap, the user receives a new actormap (a "transactormap") which is a delta to the parent actormap (either another transactormap or the root protected weak-hashtable actormap, a "whactormap").
• This transactionality is really exciting. It means that if something bad happens, we can always roll back to a safe state (or rather, never commit the unsafe state at all). In the default vat, if a message is received and an uncaught exception occurs, the promise is broken, but all the effects caused by interactions from unhandling the message are as if they never occured. (Well that is, as long as we use the "become this procedure" mechanism in Goblins to manage state! If you mutate a variable, you're on your own. A Racket #lang could prevent your users from doing such naughty things if you so care.)
• It also means that snapshotting an actormap is really easy. Elm used to advertise having a "time traveling debugger" where they showed off Mario running around, and you could reverse time to a previous state. Apparently this was removed but maybe is coming back. Anyway it's trivial to do such a thing with Goblins' actormap, and I built such a (unpublished due to being unpolished) demo.
• Most users won't work with the actormap though, they'll work with the builtin vat that takes care of all this stuff for them. You can build your own vat, or vat-like tools, though.

Anyway, all the above works and exists. Actors can even speak to each other across vats... though, what's missing so far is the ability to talk to other objects/vats on other machines. That's basically what's next on my agenda, and I know how to do it... it's just a matter of getting the stuff done.

Well, the other thing that's missing is documentation. That's competing for next thing on the agenda.

But why a synchronous game right now?

If the really exciting stuff is the distributed secure programming stuff, why did I stop to do a synchronous non-distributed game on top of Spritely Goblins? Before I plowed ahead, given that the non-distributed aspects still rest on the distributed aspects, I wanted to make sure that the fundamentals of Spritely Goblins were good.

A space shooter is simple enough to implement and using ascii art in a terminal meant I didn't need to spend too much time thinking about graphics (plus it's an interesting area that's under-explored... most terminal-based games are roguelikes or other turn-based games, not real time). Implementing it allowed me to find many areas that could be improved usability-wise in Goblins (indeed, it's been a very active month of development for Goblins). You really know what things are and aren't nice designs by using them.

It's also a great way to identify performance bottlenecks. I calculated that roughly 1 million actor invocations could happen per second on my cheapo laptop... not bad. But that was when the actors didn't update themselves; when it came to the transactional updates, I could only seem to achieve about 65k updates per second. I figured this must be the transactionality, but it turns out it wasn't; the transactionality feature is very cheap. Can you believe that I got a jump from 65k updates per second to 680k updates per second just by switching from a Racket contract to a manual predicate check? (I expected a mild performance hit for using a contract over a manual predicate, but 10x...?) (I also added a feature so you can "recklessly" commit directly to the actormap without transactions... I don't recommend this for all applications, but if you do that you can get up to about 790k updates per second... which means that transactionality adds only about a 17% overhead, which isn't even close to the 10x gap I was seeing.) Anyway, the thing that lead me to looking into that in the first place was doing an experiment where I decided I wanted to see how many objects I could have updating at once. I might not have caught it otherwise. So making a game demo is useful for that kind of thing.

I feel now that I've gotten most of the churn out of that layer of the design out of the way so that I can move forward with the design on the distributed side of things next. That allows me to have tighter focus of things in layers, and I'm happy about that.

What's next?

So with that out of the way, the next task is to work on both the mutually suspicious distributed programming environment and the documentation. I'm not sure in which order, but I guess we'll find out.

I'll do something similar with the distributed programming environment as well... I plan to write something basic which resembles a networked game at this stage to help me ensure that the components work nicely together.

In the meanwhile, Terminal Phase is very close to being a nice game to play, but I'm deciding to leave that as a funding milestone on my Patreon. This is because, as far as my technical roadmap has gone, Terminal Phase has performed the role it needs to play. But it would be fun to have, and I'm sure other people would like to play it as a finished game (heck, I would like to play it as a finished game), but I'd like to know... do people actually care enough about free software games? About this direction of work? Am I on the right track? Not to mention that funding this work is also simply damn hard.

But, at the time of writing we're fairly close, (about 85% of the way there), so maybe it will happen. If it sounds fun to you, maybe pitch in.

But one way or another, I'll have interesting things to announce ahead. Stay tuned here, or follow me on the fediverse or on Twitter if you so prefer.

Onwards and upwards!

NOTE: This update also appeared on my Patreon account. If you're reading the below and it sounds like I'm doing a lot of useful work, consider becoming a patron!

Hello all! It's been a couple of months since I've gotten out an update; much has happened.

First of all, ActivityPub Conf happened and was a big success! The video team got things recorded an uploaded so you can watch talks from the event, including my keynote (the audio quality is a bit messed up on this one, the others are better) and Mark Miller's keynote. The other talks were all also very excellent; I'm not going to iterate them all here because you can already go watch them! I think you will find there are many thematic threads between the videos.

We had about 40 people at the event; the first day was spent on talks and the second day was an "unconference" where groups self-organized to discuss various topics of mutual interest. One common thread was about the kinds of directions I've been pushing for in Spritely: distributed encrypted storage (Datashards, with Serge Wroclawski leading the conversation on that), object capabilities (OcapPub), stamps, etc. It was interesting to watch from the start to the end of the unconference day; particularly, the Pleroma folks were there and gave a lot of feedback. Towards the start of the day I think there was much more skepticism, but towards the end we were hearing belief and interest that these kinds of things could and should be implemented and would be of real use to fediverse participants. Lain of Pleroma in particular expressed that it helped to realize that even though I'm presenting all these ideas, they don't need to be implemented all at once; we can take them on piecemeal, and incrementalism is a perfectly valid approach. (Also "OcapPub" sounds like a new protocol, whereas it's really just a way-to-use ActivityPub mostly as it already exists. Maybe time for a new name for that?)

Anyway, ActivityPub Conf was a massive success; thank you everyone who came and participated. It's clear after APConf to me just how much of a difference getting folks together can make. For those who couldn't make it, let's thank the video team (DeeAnn Little, Sebastian Lasse, Markus Feilner) for getting those videos up!

On the topic of Datashards, we have a website and a nice logo now (courtesy of mray, who also made the ActivityPub logo). Serge Wroclawski (co-host with myself of Libre Lounge) has been increasingly helping with the project; before ActivityPub Conference and Rebooting Web of Trust we worked to make sure both of our implementations could talk to each other (Serge's Python implementation and my Racket implementation). At RWoT we showed a demo where I "beamed" the death star plans to Serge's computer. (We used the same content storage server, I uploaded the death star plans, rendered the QR code on my laptop, Serge scanned the QR code from his laptop, downloaded the file and showed off the plans from his computer... with the storage server having no idea about the contents of the data we were storing there!) People really liked that demo; we have had conversations about whether Datashards may serve as a foundational system for some other tools being made in that space; more later. In the meanwhile, I'm happy we have two applications in two different languages successfully being able to read and write each others' immutable datashards updates; the next step is making sure that mutability works the same.

Rebooting Web of Trust was also a very interesting event; the highlight being that I am now collaborating with some great folks on a secure user interfaces paper. We are taking the existing Mastodon web user interface and retooling it to reduce risks such as phishing, and open the path for more peer to peer systems (very timely, since that's the direction we want to take things). Unfortunately the amount of work to do on the paper is rather huge; it may take a while until the paper is complete. In the meanwhile, the Petnames paper has turned out to be a pre-requisite for the secure UIs one; that paper has been nearly complete for some time so I guess I have to finish the work. I recently added some new UI mockups to it, but there is still more to do.

Now that these conferences are over, I am putting time towards Spritely's core vision again: distributed virtual worlds. The foundational layer for that is Spritely Goblins, an ocap-secure distributed programming environment on top of Racket. I really enjoy hacking on this and I am happy to get time back to working on it. A topic of discussion came up between myself and Mark Miller at Rebooting Web of Trust though; am I unnecessarily duplicating effort between myself and the Agoric folks? In particular, they are building something equivalent (and arguably more featureful) to Spritely Goblins named SwingSet. This would run on top of Javascript rather than Racket/Lisp/Scheme. I have found that in the past I have been not very happy when working with Javascript, but Mark suggested I take a look at Agoric's Jessie subset of Javascript, which Mark described as "closer to Scheme" (not syntactically, but in terms of language-cleanliness). It does seem nicer than Javascript; when I admitted to Mark that I am addicted to parenthetical syntax, Mark posed the question about whether building a parenthetical version of Jessie would be less work than reproducing all the other things that Agoric is doing. It's a good point; I don't know. I'm unhappy with the idea of pivoting, but I do feel like it's probably true that due diligence suggests I should consider it carefully. It is true at least that I would probably reach a broader userbase more quickly with the option of Javascript syntax; it's hard for me to deny that. I will probably explore it with some smaller tests of Agoric's stuff. But in the meanwhile, I currently plan to release a very small version of the game demo using the toolkit I already am building while testing Agoric's infrastructure in parallel. I suspect we'll see the first user-visible outputs of this in early 2020.

There have been four new Libre Lounge episodes since my last update. That's still quite a few episodes to listen to, but slower than we previously were updating; all the travel is to blame. However that is settling down and I think we'll be updating more frequently soon. Even so, we have been updating!

In addition to all this, I suspect there will be at least two major announcements in the coming months; stay tuned. Work has already occured on both, but I can only say so much right now.

Thanks to everyone who has supported my work. I work much more than full time in the cause of advancing user freedom; it's not easy to fund this work. I appreciate all of you who are giving what you can.

Now, back to work!

Good news everyone! The speaker list for ActivityPub Conf 2019 is here! (In this document, below, but also in ODT and PDF formats.)

(Bad news everyone: registration is closed! We're now at 40 people registered to attend. However, we do aim to be posting recordings of the event afterwards if you couldn't register in time.)

But, just in case you'd rather see the list of speakers on a webpage rather than download a document, here you go:

Keynote: Mark Miller, “Architectures of Robust Openness”

Description coming soon! But we're very excited about Mark Miller keynoting.

Keynote: Chris Lemmer-Webber, “ActivityPub: past, present, future”

This talk gives an overview of ActivityPub: how did we get to this point? Where are we now? Where do we need to go? We'll paint a chart from past to a hopeful future with better privacy, richer interactions, and more security and control for our users.

Matt Baer, “Federated Blogging with WriteFreely”

We're building out one idea of what federated blogging could look like with separate ActivityPub-powered platforms, WriteFreely and Read.as -- one for writing and and one for reading. Beyond the software, we're also offering hosting services and helping new instances spring up to make community-building more accessible, and get ActivityPub-powered software into more hands. In this talk I'll go over our approach so far and where we're headed next.

Caleb James DeLisle, “The case for the unattributed message”

Despite it's significant contribution to internet culture, the archetype of the anonymous image board has been largely ignored by protocol designers. Perhaps the reason for this is because it's all too easy to conflate unattributed speech with unmoderated speech, which has shown itself to be a dead end. But as we've seen from Twitter and Facebook, putting a name on everything hasn't actually worked that well at improving the quality of discourse, but what it does do is put already marginalized people at greater risk.

What I credit as one of the biggest breakthroughs of the fediverse has been the loose federation which allows a person to choose their moderator, completely side stepping the question of undemocratic censorship vs. toxic free speech. Now I want to start a conversation about how we might marry this powerful moderation system to a forum which divorces the expression of thought from all forms of identity.

Cristina DeLisle, “OSS compliance with privacy by default and design”

Privacy is becoming more and more central in shaping the future of tech and the data protection legislation has contributed significantly to making this happen. Privacy by default and design are core principles that are fundamental to how software should be envisioned. The GDPR that came into the spotlight has a strong case to become a standard even outside European borders, influencing the way we protect personal data. However its impact might be, its implementation is still in its infancy. OSS has found itself facing the situation and one aspect which is particularly interesting on the tech side is how to incorporate the principles of privacy by default and design into the software that we build.

This talk is going to be an overview of how the GDPR has impacted FOSS communities, what do we mean by privacy by default and by design, how could we envision them applied in our OSS. It will bring examples from which we might find something interesting to learn from, regardless if we are looking at them as mistakes, best practices or just ways of doing things.

Michael Demetriou, “I don't know what I'm talking about: a newbie's introduction to ActivityPub”

I have just started my development journey in ActivityPubLand and I hope to have a first small application ready before ActivityPubConf. I was thinking that since I have close to zero experience with ActivityPub development, I could document my first month of experience, describe the onboarding process and point out useful resources and common pitfalls. In the end I can showcase what I've done during this period.

Luc Didry, “Advice to new fediverse administrators and developers”

Hosting an ActivityPub service is not like hosting another service… and it's the same for developing ActivityPub software. Here is some advice based on Framasoft's experience (we host a Mastodon instance and develop two ActivityPub software: PeerTube and Mobilizon – the last one is not yet out), errors and observations.

Maloki, “Is ActivityPub paving the way to web 3.0?”

A talk about how we're walking away from Web 2.0, and paving the way to Web 3.0 with ActivityPub development. We'll discuss what this could mean for the future of the web, we'll look at some of the history of the web, and also consider the social implications moving forward.

Pukkamustard, “The Semantic Social Network”

ActivityPub uses JSON-LD as serialization. This means [@context]{.title-ref} field all over the place. But really there is more behind this: ActivityPub speaks Linked Data. In this talk we would like to show what this means and how this can be used to do cool things. We might even convince you that the Fediverse is a huge distributed graph that could be queried in very interesting ways - that the Fediverse is a Semantic Social Network.

Schmittlauch, “Decentralised Hashtag Search and Subscription in Federated Social Networks”

Hashtags have become an important tool for organising topic-related posts in all major social networks, even having managed to spark social movements like #MeToo. In federated social networks, unfortunately so far the view on all posts of a hashtag is fragmented between instances.

For a student research paper I came up with an architecture for search and subscription of hashtag-posts in federated social networks. This additional backend for instances augments the Fediverse with a little bit of P2P technology.

As this architecture is still at a conceptual stage, after presenting my work I'd like to gather ideas and feedback from various Fediverse stakeholders: What do global hashtags mean for marginalised people and moderation, are they more a tool of empowerment or of harassment? How can this concept be represented in the ActivityPub protocol? And what stories do server devs have to tell about common attack scenarios?

Serge Wroclawski, “Keeping Unwanted Messages off the Fediverse”

Spam, scams and harassment pose a threat to all social networks, including the Fediverse. In this talk, we discuss a multilayered approach to mitigating these threats. We explore spam mitigation techniques of the past as well as new techniques such as OcapPub and Postage.

I am extremely pleased to announce that Mark S. Miller is keynoting at ActivityPub Conf 2019!

It's hard for me to understate how huge this is. Mark S. Miller works at Agoric which is leading the way on modern application of object capabilities, which is convenient, since exploration of how to apply object capabilities to federated social networks is a major topic of interest on the fediverse.

But just leaving it at that would be leaving out too much. We can trace Mark's work back the Agoric papers in 1988 which laid out the vision for a massive society and economy of computing agents. (And yes, that's where the Agoric company got its name from.)

For 30 years Mark has been working towards that vision, and social networks continued to intersect with its work. In the late 1990s Mark was involved in a company working on the game Electric Communities Habitat (it's hard to find information on it, but here's a rare video of it in action). (Although Mark Miller didn't work on it, Electric Communities Habitat has its predecessor in Lucasfilm's Habitat, which it turns out was a graphical multiplayer game which ran on the Commodore 64!(!!!) You can see the entertaining trailer for this game... keep in mind, this was released in 1986!)

People who have read my blog before may know that I've talked about building secure social spaces as virtual worlds: part of the reason I know it is possible is that Electric Communities Habitat for the large part built it and proved the ideas possible. Electric Communities the company did not survive, but the ideas lived on in the E programming language, which I like to describe as "the most interesting and important programming language you may have never heard of".

While the oldschool design of the website may give you the impression that the ideas there are out of date, time and time again I've found that the answers to my questions about how to build things have all been found on erights.org and in Mark Miller's dissertation.

Mark's work hasn't stopped there. Many good ideas in Javascript (such as its promises system) were largely inspired from Mark's work on the E programming language (Mark joined the standardization process of Javascript to make it be possible to build ocap-safe systems on it), and... well, I can go on and on.

Instead, I'm going to pause and say that I'm extremely excited that Mark has agreed to come to ActivityPub Conf to help introduce the community to the ideas in object capabilities. I hope the history I laid out above helps make it clear that the work to coordinate cooperative behavior amongst machines overlaps strongly with our work in the federated social web of establishing cooperative behavior amongst communities of human beings. I look forward to Mark helping us understand how to apply these ideas to our space.

Has this post got you excited? At the time of me writing this, there's still space at ActivityPub Conf 2019, and there's still time (until Monday July 29th) to submit talks. See the conference announcement for more details, and hope to see you there!

EDIT: I incorrectly cited Mark Miller originally as being involved in Lucasfilm's Habitat; fixed and better explained its history.

{.align-center}

UPDATE: As of August 5th, registrations have now filled up! See all of you who registered at ActivityPub Conf!

That's right! We're hosting the first ever ActivityPub Conf. It's immediately following Rebooting Web of Trust in Prague.

There's no admission fee to attend. (Relatedly, the conference is kind of being done on the cheap, because it is being funded by organizers who are themselves barely funded.) The venue, however, is quite cool: it's at the DOX Centre for Contemporary Art, which is itself exploring the ways the digital world is affecting our lives.

If you plan on attending (and maybe also speaking), you should get in your application soon (see the flier for details). We've never done one of these, and we have no idea what the response will be like, so this is going to be a smaller gathering (about 40 people). In some ways, it will be somewhere between a conference and a gathering of people-who-are-interested-in-activitypub.

As said in the flier, by attending, you are agreeing to the code of conduct, so be sure to read that.

The plan is that the first day will be talks (see the flier above for details on how to apply as a speaker) and the second day will be an unconference, with people splitting off into groups to work through problems of mutual interest.

Applications for general admission are first-come-first-serve. Additionally, we have reserved some slots for speakers specifically; the application to get in submissions for talks is 1 week from today (July 29th). We are hoping for and encouraging a wide range of participant backgrounds.

Hope to see you in Prague!

A little over a decade ago, there were some popular blogposts about whether Ruby was an acceptable Lisp or whether even Lisp was an acceptable Lisp. Peter Norvig was also writing at the time introducing Python to Lisp programmers. Lisp, those in the know knew, was the right thing to strive for, and yet seemed unattainable for anything aimed for production since the AI Winter shattered Lisp's popularity in the 80s/early 90s. If you can't get Lisp, what's closest thing you can get?

This was around the time I was starting to program; I had spent some time configuring my editor with Emacs Lisp and loved every moment I got to do it; I read some Lisp books and longed for more. And yet when I tried to "get things done" in the language, I just couldn't make as much headway as I could with my preferred language for practical projects at the time: Python.

Python was great... mostly. It was easy to read, it was easy to write, it was easy-ish to teach to newcomers. (Python's intro material is better than most, but my spouse has talked before about some major pitfalls that the Python documentation has which make getting started unnecessarily hard. You can hear her talk about that at this talk we co-presented on at last year's RacketCon.) I ran a large free software project on a Python codebase, and it was easy to get new contributors; the barrier to entry to becoming a programmer with Python was low. I consider that to be a feature, and it certainly helped me bootstrap my career.

Most importantly of all though, Python was easy to pick up and run with because no matter what you wanted to do, either the tools came built in or the Python ecosystem had enough of the pieces nearby that building what you wanted was usually fairly trivial.

But Python has its limitations, and I always longed for a lisp. For a brief time, I thought I could get there by contributing to the Hy project, which was a lisp that transformed itself into the Python AST. "Why write Python in a syntax that's easy to read when you could add a bunch of parentheses to it instead?" I would joke when I talked about it. Believe it or not though, I do consider lisps easier to read, once you are comfortable to understand their syntax. I certainly find them easier to write and modify. And I longed for the metaprogramming aspects of Lisp.

Alas, Hy didn't really reach my dream. That macro expansion made debugging a nightmare as Hy would lose track of where the line numbers are; it wasn't until that when I really realized that without line numbers, you're just lost in terms of debugging in Python-land. That and Python didn't really have the right primitives; immutable datastructures for whatever reason never became first class, meaning that functional programming was hard, "cons" didn't really exist (actually this doesn't matter as much as people might think), recursive programming isn't really as possible without tail call elimination, etc etc etc.

But I missed parentheses. I longed for parentheses. I dreamed in parentheses. I'm not kidding, the only dreams I've ever had in code were in lisp, and it's happened multiple times, programs unfolding before me. The structure of lisp makes the flow of code so clear, and there's simply nothing like the comfort of developing in front of a lisp REPL.

Yet to choose to use a lisp seemed to mean opening myself up to eternal yak-shaving of developing packages that were already available on the Python Package Index or limiting my development community an elite group of Emacs users. When I was in Python, I longed for the beauty of a Lisp; when I was in a Lisp, I longed for the ease of Python.

All this changed when I discovered Racket:

• Racket comes with a full-featured editor named DrRacket built-in that's damn nice to use. It has all the features that make lisp hacking comfortable previously mostly only to Emacs users: parenthesis balancing, comfortable REPL integration, etc etc. But if you want to use Emacs, you can use racket-mode. Win-win.
• Racket has intentionally been built as an educational language, not unlike Python. One of the core audiences of Racket is middle schoolers, and it even comes with a built-in game engine for kids. (The How to Design Programs prologue might give you an introductory taste, and Realm of Racket is a good book all about learning to program by building Racket games.)
• My spouse and I even taught classes about how to learn to program for humanities academics using Racket. We found the age-old belief that "lisp syntax is just too hard" is simply false; the main thing that most people lack is decent lisp-friendly tooling with a low barrier to entry, and DrRacket provides that. The only people who were afraid of the parentheses turned out to be people who already knew how to program. Those who didn't even praised the syntax for its clarity and the way the editor could help show you when you made a syntax error (DrRacket is very good at that). "Lisp is too hard to learn" is a lie; if middle schoolers can learn it, so can more seasoned programmers.
• Racket might even be more batteries included than Python. At least all the batteries that come included are generally nicer; Racket's GUI library is the only time I've ever had fun in my life writing GUI programs (and they're cross platform too). Constructing pictures with its pict library is a delight. Plotting graphs with plot is an incredible experience. Writing documentation with Scribble is the best non-org-mode experience I've ever had, but has the advantage over org-mode in that your document is just inverted code. I could go on. And these are just some packages bundled with Racket; the Package repository contains much more.
• Racket's documentation is, in my experience, unparalleled. The Racket Guide walks you through all the key concepts, and the Racket Reference has everything else you need.
• The tutorials are also wonderful; the introductory tutorial gets your feet wet not through composing numbers or strings but by building up pictures. Want to learn more? The next two tutorials show you how to build web applications and then build your own web server.
• Like Python, even though Racket has its roots in education, it is more than ready for serious practical use. These days, when I want to build something and get it done quickly and efficiently, I reach for Racket first.

Racket is a great Lisp, but it's also an acceptable Python. Sometimes you really can have it all.

I've been hacking away at Spritely (see previously). Recently I've been making progress on both the actor model (goblins and its rewrite goblinoid) as well as the storage layers (currently called Magenc and Crystal, but we are talking about probably renaming the both of them into a suite called "datashards"... yeah, everything is moving and changing fast right now.)

In the #spritely channel on freenode a friend asked, what is the big picture idea here? Both the actor model layer and the storage layer describe themselves as using "capabilities" (or more precisely "object capabilities" or "ocaps") but they seem to be implemented differently. How does it all tie together?

A great question! I think the first point of confusion is that while both follow the ocap paradigm (which is to say, reference/possession-based authority... possessing the capability gives you access, and it does not matter what your identity is for the most part for access control), they are implemented very differently because they are solving different problems. The storage system is based on encrypted, persistent data, with its ideas drawn from Tahoe-LAFS and Freenet, and the way that capabilities work is based on possession of cryptographic keys (which are themselves embedded/referenced in the URIs). The actor model, on the other hand, is based on holding onto a reference to a unique, unguessable URL (well, that's a bit of an intentional oversimplification for the sake of this explaination but we'll run with it) where the actor at that URL is "live" and communicated with via message passing. (Most of the ideas from this come from E and Waterken.) Actors are connected to each other over secure channels to prevent eavesdropping or leakage of the capabilities.

So yeah, how do these two seemingly very different layers tie together? As usual, I find that I most easily explain things via narrative, so let's imagine the following game scenario: Alice is in a room with a goblin. First Alice sees the goblin, then Alice attacks the goblin, then the goblin and Alice realize that they are not so different and become best friends.

The goblin and Alice both manifest in this universe as live actors. When Alice walks into the room (itself an actor), the room gives Alice a reference to the goblin actor. To "see" the goblin, Alice sends a message to it asking for its description. It replies with its datashards storage URI with its 3d model and associated textures. Alice can now query the storage system to reconstruct these models and textures from the datashards storage systems she uses. (The datashards storage systems themselves can't actually see the contents if they don't have the capability itself; this is much safer for peers to help the network share data because they can help route things through the network without personally knowing or being responsible for what the contents of those messages are. It could also be possible for the goblin to provide Alice with a direct channel to a storage system to retrieve its assets from.) Horray, Alice got the 3d model and images! Now she can see the goblin.

Assuming that the goblin is an enemy, Alice attacks! Attacking is common in this game universe, and there is no reason necessarily to keep around attack messages, so sending a message to the goblin is just a one-off transient message... there's no need to persist it in the storage system.

The attack misses! The goblin shouts, "Wait!" and makes its case, that both of them are just adventurers in this room, and shouldn't they both be friends? Alice is touched and halts her attack. These messages are also sent transiently; while either party could log them, they are closer to an instant messenger or IRC conversation rather than something intended to be persisted long-term.

They exchange their mailbox addresses and begin sending each other letters. These, however, are intended to be persisted; when Alice receives a message from the goblin in her mailbox (or vice versa), the message received contains the datashards URI to the letter, which Alice can then retrieve from the appropriate store. She can then always refer to this message, and she can choose whether or not to persist it locally or elsewhere. Since the letter has its own storage URI, when Alice constructs a reply, she can clearly mark that it was in reference to the previous letter. Even if Alice or the goblin's servers go down, either can continue to refer to these letters. Alice and the goblin have the freedom to choose what storage systems they wish, whether targeted/direct/local or via a public peer to peer routing system, with reasonable assumptions (given the continued strength of the underlying cryptographic algorithms used) that the particular entities storing or forwarding their data cannot read its content.

And so it is: live references of actors are able to send live, transient messages, but can only be sent to other actors whose (unguessable/unforgeable) address you have. This allows for highly dynamic and expressive interactions while retaining security. Datashards URIs allow for the storage and retrieval of content which can continue to be persisted by interested parties, even if the originating host goes down.

There are some things I glossed over in this writeup. The particular ways that the actors' addresses and references work is one thing (unguessable http based capability URLs on their own have leakage problems due to the way various web technologies are implemented, and not even every actor reference needs to be a long-lived URI; see CapTP for more details), how to establish connections between actor processes/servers (we can reuse TLS, or even better, something like tor's onion services), so are how interactions such as fighting can be scoped to a room (this paper explains how), as well as how we can map human meaningful names onto unguessable identifiers (the answer there is petnames). But I have plans for this and increasing confidence that it will come together... I think we're already on track.

Hopefully this writeup brings some clarity on how some of the components will work together, though!

Approximately a month ago was Morgan and I's 10 year wedding anniversary. To commemorate that, and as a surprise gift, I made the above ascii art and animation.

Actually, it's not just an animation, it's a program, and one you can run. As a side note, I originally thought I'd write up how I made it, but I kept procrastinating on that and it lead me to putting off writing this post for about a month. Oh well, all I'll say for now is that it lead to a major rewrite of one of the main components of Spritely. But that's something to speak of for another time, I suppose.

Back to the imagery! Morgan was surprised to see the animation, and yet the image itself wasn't a surprise. That's because the design is actually built off of one we collaborated on together:

I did the sketch for it and Morgan embroidered it. The plan is to put this above the tea station we set up in the reading area of our house.

The imagery and phrasing captures the philosophy of Morgan and I's relationship. We're both weird and deeply imperfect people, maybe even in some ways broken. But that's okay. We don't expect each other to change or become something else... we just try to become the best weird pairing we can together. I think that strategy has worked out for us.

Thanks for all the happy times so far, Morgan. I look forward to many weird years ahead.

Did I somehow not blog here that I started co-hosting a podcast named Libre Lounge with my friend Serge Wroclawski? We're talking about all sorts of topics facing user freedom. Take a look at the archive and you might find something you like!

At the time of writing we've also had on one guest, Karen Sandler. This was a really nice treat since the show Free as in Freedom, which Karen co-hosts, is clearly a big influence on Libre Lounge. Anyway, we plan to have on lots more guests in the future.

Hope you enjoy!