I've been hacking away at Spritely (see previously). Recently I've been making progress on both the actor model (goblins and its rewrite goblinoid) as well as the storage layers (currently called Magenc and Crystal, but we are talking about probably renaming the both of them into a suite called "datashards"... yeah, everything is moving and changing fast right now.)

In the #spritely channel on freenode a friend asked, what is the big picture idea here? Both the actor model layer and the storage layer describe themselves as using "capabilities" (or more precisely "object capabilities" or "ocaps") but they seem to be implemented differently. How does it all tie together?

A great question! I think the first point of confusion is that while both follow the ocap paradigm (which is to say, reference/possession-based authority... possessing the capability gives you access, and it does not matter what your identity is for the most part for access control), they are implemented very differently because they are solving different problems. The storage system is based on encrypted, persistent data, with its ideas drawn from Tahoe-LAFS and Freenet, and the way that capabilities work is based on possession of cryptographic keys (which are themselves embedded/referenced in the URIs). The actor model, on the other hand, is based on holding onto a reference to a unique, unguessable URL (well, that's a bit of an intentional oversimplification for the sake of this explaination but we'll run with it) where the actor at that URL is "live" and communicated with via message passing. (Most of the ideas from this come from E and Waterken.) Actors are connected to each other over secure channels to prevent eavesdropping or leakage of the capabilities.

So yeah, how do these two seemingly very different layers tie together? As usual, I find that I most easily explain things via narrative, so let's imagine the following game scenario: Alice is in a room with a goblin. First Alice sees the goblin, then Alice attacks the goblin, then the goblin and Alice realize that they are not so different and become best friends.

The goblin and Alice both manifest in this universe as live actors. When Alice walks into the room (itself an actor), the room gives Alice a reference to the goblin actor. To "see" the goblin, Alice sends a message to it asking for its description. It replies with its datashards storage URI with its 3d model and associated textures. Alice can now query the storage system to reconstruct these models and textures from the datashards storage systems she uses. (The datashards storage systems themselves can't actually see the contents if they don't have the capability itself; this is much safer for peers to help the network share data because they can help route things through the network without personally knowing or being responsible for what the contents of those messages are. It could also be possible for the goblin to provide Alice with a direct channel to a storage system to retrieve its assets from.) Horray, Alice got the 3d model and images! Now she can see the goblin.

Assuming that the goblin is an enemy, Alice attacks! Attacking is common in this game universe, and there is no reason necessarily to keep around attack messages, so sending a message to the goblin is just a one-off transient message... there's no need to persist it in the storage system.

The attack misses! The goblin shouts, "Wait!" and makes its case, that both of them are just adventurers in this room, and shouldn't they both be friends? Alice is touched and halts her attack. These messages are also sent transiently; while either party could log them, they are closer to an instant messenger or IRC conversation rather than something intended to be persisted long-term.

They exchange their mailbox addresses and begin sending each other letters. These, however, are intended to be persisted; when Alice receives a message from the goblin in her mailbox (or vice versa), the message received contains the datashards URI to the letter, which Alice can then retrieve from the appropriate store. She can then always refer to this message, and she can choose whether or not to persist it locally or elsewhere. Since the letter has its own storage URI, when Alice constructs a reply, she can clearly mark that it was in reference to the previous letter. Even if Alice or the goblin's servers go down, either can continue to refer to these letters. Alice and the goblin have the freedom to choose what storage systems they wish, whether targeted/direct/local or via a public peer to peer routing system, with reasonable assumptions (given the continued strength of the underlying cryptographic algorithms used) that the particular entities storing or forwarding their data cannot read its content.

And so it is: live references of actors are able to send live, transient messages, but can only be sent to other actors whose (unguessable/unforgeable) address you have. This allows for highly dynamic and expressive interactions while retaining security. Datashards URIs allow for the storage and retrieval of content which can continue to be persisted by interested parties, even if the originating host goes down.

There are some things I glossed over in this writeup. The particular ways that the actors' addresses and references work is one thing (unguessable http based capability URLs on their own have leakage problems due to the way various web technologies are implemented, and not even every actor reference needs to be a long-lived URI; see CapTP for more details), how to establish connections between actor processes/servers (we can reuse TLS, or even better, something like tor's onion services), so are how interactions such as fighting can be scoped to a room (this paper explains how), as well as how we can map human meaningful names onto unguessable identifiers (the answer there is petnames). But I have plans for this and increasing confidence that it will come together... I think we're already on track.

Hopefully this writeup brings some clarity on how some of the components will work together, though!

Approximately a month ago was Morgan and I's 10 year wedding anniversary. To commemorate that, and as a surprise gift, I made the above ascii art and animation.

Actually, it's not just an animation, it's a program, and one you can run. As a side note, I originally thought I'd write up how I made it, but I kept procrastinating on that and it lead me to putting off writing this post for about a month. Oh well, all I'll say for now is that it lead to a major rewrite of one of the main components of Spritely. But that's something to speak of for another time, I suppose.

Back to the imagery! Morgan was surprised to see the animation, and yet the image itself wasn't a surprise. That's because the design is actually built off of one we collaborated on together:

I did the sketch for it and Morgan embroidered it. The plan is to put this above the tea station we set up in the reading area of our house.

The imagery and phrasing captures the philosophy of Morgan and I's relationship. We're both weird and deeply imperfect people, maybe even in some ways broken. But that's okay. We don't expect each other to change or become something else... we just try to become the best weird pairing we can together. I think that strategy has worked out for us.

Thanks for all the happy times so far, Morgan. I look forward to many weird years ahead.

Did I somehow not blog here that I started co-hosting a podcast named Libre Lounge with my friend Serge Wroclawski? We're talking about all sorts of topics facing user freedom. Take a look at the archive and you might find something you like!

At the time of writing we've also had on one guest, Karen Sandler. This was a really nice treat since the show Free as in Freedom, which Karen co-hosts, is clearly a big influence on Libre Lounge. Anyway, we plan to have on lots more guests in the future.

Hope you enjoy!

Good news everyone! I've been awarded the Samsung Stack Zero Grant. But why not quote them?

Chris Lemmer-Webber is the co-editor and co-author of the now-ubiquitous ActivityPub protocol. While it provides a great framework for creating, updating, and deleting content across applications, it doesn’t provide any standardised mechanism for secure authorisation. With Spritely, Webber will work on extending the protocol in a backward-compatible manner, while at the same time building tools and applications that showcase its use. This will enable developers to build applications that enable richer interactions through a federated standard.

This should fund my next couple of years of work on full time advancement of the fediverse.

You may remember that I've talked about Spritely before. In fact I am finally in launch-mode... I am currently sitting in a wizard's tower at a hackathon, getting out the first release of Golem, a Spritely artifact.

Anyway, I'll be at FOSDEM 2019 giving a talk and on a panel. And after that I'll be speaking at CopyleftConf. Maybe I'll see you?

More news soon...

In my last post I talked about launching the Spritely project in all its ambitiousness. As I said, I plan on releasing it as a series of "demos". What I mean by "demo" is fairly loose: things you can try and see, artifacts to indicate progress as I go. Convergence slowly towards a goal.

Over the last week I released the first version of the foundation of this work, which I'm calling Goblins. v0.1, a pre-pre-alpha release, is now out the door. Goblins is an actor model implementation for the Racket programming language. I think if you know some Racket, then the Goblins documentation should be fairly approachable, and I'd love to hear feedback. Not everything is documented, but it should give you a nice taste of what the core ideas are. (Astute readers may observe that Goblins' name shares striking resemblance to another project I have worked on; this is no coincidence and is indeed a hint as to what I think the future direction of that area of work is.)

Most live distributed systems are in some way a language-level actor model implementation, but knowing that you're an actor model implementation may change how you do things. (ActivityPub is itself an actor model protocol, though I think not all implementors realize that.) Goblins is the third generation of an actor model I've written, the previous ones being XUDD and 8sync. (Maybe if you count the big rewrite I did of 8sync to run it on top of fibers, it could be the fourth generation.)

If you read the Spritely blogpost you'll know that distributed games are part of the long con here. This isn't the first time I've written a game system on top of one of these actor model implementations; in fact if you scroll down on 8sync website you'll (currently) see a video of me presenting on Mudsync, a multi-user dungeon written in 8sync, where in the talk instead of slides I had rooms, and the audience was moving around in the world and I was changing it in response to their suggestions.

But the drive of the actor model connected to social distributed games goes back further for me. I initially got interested in an actor model the first time I tried to work on the whole multiplayer game front about a decade ago. A friend and I spent a couple hours experimenting with threads and locks and after the first demo it was obvious that was never going to work. I remembered I had been told by a former coworker that they had used a system that had an actor model + coroutines as its basis; I didn't know what either of those things were yet, but as I looked into them it became clear to me that this was the right route. But one piece since then until recently remained a mystery to me, which was how to manage the complex web of authorization between the objects.

Last year at TPAC, the World Wide Web consortium's conference, I was staying with some people who asked me what I would do if I had unlimited time to work on whatever I wanted. I sheepishly admitted to my dream of a federated social web as a distributed game, and expected to be laughed out of the room. I was surprised to find out that nearly everyone I was working with had some background in this same work and dream. This gave me some hope that I wasn't completely foolish, and the observation that even failure in persuing this ambitious dream would probably result in useful artifacts.

And last year at Rebooting Web of Trust, I hit a big breakthrough. I had the chance to meet Mark S. Miller in person, someone who's work I read but barely understood at the time, and this meeting in some ways changed the direction of my career. For one thing, it lead to us collaborating on the foundation for the ocap-ld specification, which came out of a paper we collaborated on from that event (also available in nicely formatted PDF form). But what I also discovered was that Mark Miller and company had actually built the distributed and secure social network dream in the late 1990s and called it Electric Communities (rare video of it being demonstrated, and a rare video of some of the out-there ideas). What I found out was that they had solved most of the remaining questions about authorization and security through object capabilities, and that was through a pile of object capability patterns built on top of the actor model. While Electric Communities did not survive, its foundation of the the "E" programming language lived on, pulling in many of its good ideas with it. I proceeded to read as much documentation about E and surrounding capability patterns as I could, and this has majorly influenced Goblins' design. (Don't underestimate the depth of ideas on erights.org just because it looks like an ancient Geocities page... it did come from that era after all...)

Another key insight came from discovering the Object-Capability Security in Virtual Environments paper. I won't go into details on this except to say that it's a must-read for this problem domain, with the caveat that it pulls in a lot of terminology baggage from the object capability (aka ocap) community. The ocap community is one of those groups that has so many of the right ideas but which unfortunately is also drowning in its own verbiage, despite true efforts to make the ideas more accessible. We (and I guess I'm part of this group now, so "we" is appropriate) need to do better, but it isn't easy.

Goblins is still in its very early days. Its api is unstable and it surely has bugs (this one in particular I need to deal with, and which even exposed an underlying bug in Racket, which did get fixed quickly because the Racket community is amazing). Please don't build anything production-oriented on top of it yet. That said, I'd love users to try things. Check out the docs, see if they're up your alley.

If you like this kind of work and want to see more of it, consider donating; I don't have a corporate sponsor, so progress is funded by people like you. And keep watching here; more Spritely demos/artifacts/releases on the horizon.

If you follow me on the fediverse, maybe you already know. I've sent an announcement to my work that I am switching to doing a project named Spritely on my own full time. (Actually I'm still going to be doing some contracting with my old job, so I'll still have some income, but I'll be putting a full 40 hours a week into Spritely.)

tl;dr: I'm working on building the next generation of the fediverse as a distributed game. You can support this work if you so wish.

What on earth is Spritely?

"Well, vaporware currently", has been my joke since announcing it, but the plans, and even some core components, are starting to congeal, and I have decided it's time to throw myself fully into it.

But I still haven't answered the question, so I'll try to do so in bullet points. Spritely:

• Aims to bring stronger user security, better anti-abuse tooling, stronger resistance against censorship, and more interesting interactions to users of the fediverse.
• Is based on the massively popular ActivityPub standard (which I co-authored, so I do know a thing or two about this).
• Aims to transform distributed social networks into distributed social games / virtual worlds. The dreams of the 90s are alive in Spritely.
• Recognizes that ActivityPub is based on the actor model, and a pure version of the actor model is itself already a secure object capability system, so we don't have to break the spec to gain those powers... just change the discipline of how we use it.
• Will be written in Racket.
• Is an umbrella project for a number of modular tools necessary to get to this goal. The first, an object capability actor model system for Racket named Goblins, should see its first public release in the next week or two.
• And of course it will be 100% free/libre/open source software.

That's a lot to unpack, and it also may sound overly ambitious. The game part in particular may sound strange, but I'll defend it on three fronts. First, not too many people run federated social web servers, but a lot of people run Minecraft servers... lots of teenagers run Minecraft servers... and it's not because Minecraft has the best graphics or the best fighting (it certainly doesn't), it's because Minecraft allows you to build a world together with your friends. Second, players of old MUDs, MOOs, MUSHes and etc from the 90s may recognize that modern social networks are structurally degenerate forms of the kinds of environments that existed then, but contemporary social networks lack the concept of a sense of place and interaction. Third, many interesting projects (Python's Twisted library, Flickr, much of object capability security patterns) have come out of trying to build such massively multiplayer world systems. Because of this last one in particular, I think that shooting for the stars means that if we don't make it we're likely to at least make the moon, so failure is okay if it means other things come out of it. (Also, four: it's a fun and motivating use case for me which I have explored before.)

To keep Spritely from being total vaporware, the way I will approach the project is by regularly releasing a series of "demos", some of which may be disjoint, but will hopefully increasingly converge on the vision. Consider Spritely a skunkworks-in-the-public-interest for the federated social web.

But why?

Standardizing ActivityPub was a much more difficult effort than anticipated, but equally or more so more successful than I expected (partly due to Mastodon's adoption launching it past the sound barrier). In that sense this is great news. We now have dozens of projects adopting it, and the network has (at last I looked) over 1.5 million registered users (which isn't the same as active users).

So, mission accomplished, right? Well, there are a few things that bother me.

• The kind of rich interactions one can do are limited by a lack of authorization policy. Again, I believe object capabilities provide this, but it's not well explained to the public how to use it. (By contrast, Access Control Lists and friends are absolutely the wrong approach.)
• Users are currently insufficiently protected from spam, abuse, and harassment while at the same time administrators are overwhelmed. This is leading a number of servers to move to a whitelisting of servers, which both re-centralizes the system and prioritizes big instances over smaller instances (it shouldn't matter what instance size you're on; arguably we should be encouraging smaller ones even). There are some paths forward, and I will hint at just one: what would happen if instead of one inbox, we had multiple inboxes? If I don't know you, you can access me via my public inbox, but maybe that's heavily moderated or you have to pay "postage". If I do know you, you might have an address with more direct access to me.
• Relatedly, contemporary fediverse interfaces borrow from surveillance-capitalism based popular social networks by focusing on breadth of relationships rather than depth. Ever notice how the first thing Twitter shows you when you hover over a person's face is how many followers they have? I don't know about you, but I immediately compare that to my own follower count, and I don't even want to. This encourages high school popularity contest type bullshit, and it's by design. What if instead of focusing on how many people we can connect to we instead focused on the depth of our relationships? Much of the fediverse has imported "what works" directly from Facebook and Twitter, but I'd argue there's a lot we can do if we drop the assumption that this is the ideal starting base.
• The contemporary view in the fediverse is that social scoping is like Python scoping: locals (instance) and globals (federation). Instance administrators are even encouraged to set up to run communities based on a specific niche, which is a nice reason to motivate administrators but it causes problems: even small differences between servers' expected policies often result in servers banning each other entirely. (Sometimes this is warranted, and I'm not opposed to moderation but rather looking for more effective forms of it.) Yet most of us are one person but part of many different communities with different needs. For instance, Alice may be a computer programmer, a tabletop game enthusiast, a fanfiction author, and a member of her family. In each of those settings she may present herself differently and also have different expectations of what is acceptable behavior. Alice should not need multiple accounts for this on different servers, so it would seem the right answer for community gathering is closer to something like mailing lists. What is acceptable at the gaming table may not be acceptable at work, and what happens on the fanfiction community perhaps does not need to be shared with one's family, and each community should be empowered to moderate appropriately.
• I'd like to bridge the gap between peer to peer and federated systems. One hint as to how to do this: what happens when you run ActivityPub servers over Tor onion services or I2P? What if instead of our messages living at http addresses that could down, they could be securely addressed by their encrypted contents?
• Finally, I will admit the most urgent reason for these concerns... I'm very concerned politically about the state of the world and what I see as increasing authoritarianism and flagrant violations of human rights. I have a lot of worry that if we don't normalize use of decentralized and secure private systems, we will lose the ability to host them, though we've never needed them more urgently.

There are a lot of opportunities, and a lot of things I am excited about, but I am also afraid of inaction and how many regrets I will have if I don't try. I have the knowledge, the privilege, and the experience to at least attempt to make a dent in some of these things. I might not succeed. But I should try.

Who's going to pay for all this?

I don't really have a funding plan, so I guess this is kind of a non-answer. However, I do have a Patreon account you could donate to.

But should you donate? Well, I dunno, I feel like that's your call. Certainly many people are in worse positions than I am; I have a buffer and I still am doing some contracting to keep myself going for a while. Maybe you know people who need the money more than I do, or maybe you need it yourself. If this is the case, don't hesitate: take care of yourself and your loved ones first.

That said, FOSS in general has the property of being a public good but tends to have a free rider problem. While we did some fundraising for some of this stuff a few years ago, I gave the majority of the money to other people. Since then I've been mostly funding work on the federated social web myself in one way or another, usually by contracting on unrelated or quasi-related things to keep myself above the burn rate. I have the privilege and ability to do it, and I believe it's critical work. But I'd love to be able to work on this with focus, and maybe get things to the point to pull in and pay other people to help again. Perhaps if we reach that point I'll look at putting this work under a nonprofit. I do know I'm unwilling to break my FOSS principles to make it happen.

Anyway... you may even still be skeptical after reading all this about whether or not I can do it. I don't blame you... even I'm skeptical. But I'll try to convince you the way I'm going to convince myself: by pushing out demos until we reach something real.

Onwards and upwards!

Recently I was on a private mailing list thread where there was debate about whether or not the project should take on steps to improve diversity. One of the mailing list participants was very upset about this idea, and said that they didn't like when people accused them of the "original sin" of having white male privilege.

I suspect this is at the root of a lot of misunderstanding and frustration around the term "privilege". Privilege is not a sin... you are not a sinner for having privilege. However it is a responsibility and a debt to be repaid and corrected for, stemming from injustices in society.

A popular social narrative is that everyone has an equal place at the starting line, so the winners and losers of a race are equally based on their merit. Unfortunately this isn't true. Privilege is being able to show up at the starting line having had sleep and a good meal and the resources to train... you still worked to be able to get to the finish line and having privilege does not take that away. But if we look at the other people on the track we could see that they not only maybe didn't get enough sleep or were not able to allocate time to train (maybe they had to work multiple jobs on the side) or couldn't afford to eat as healthily. Some of them actually may even have to start back farther from the starting line, there are rocks and weeds and potholes in their paths. If we really want to treat everyone based on merit, we'd have to give everyone an equal place at the starting line, an equal track, etc. Unfortunately, due to the way the race is set up, that does mean needing to correct for some things, and it requires actual effort to repair the track.

My spouse Morgan Lemmer-Webber is an art historian and recently got into free software development (and software development in general). She has faced sexism, as all women do, her entire life, but it was immediately more visible and severe once she entered the technology space. For example, she wrote a web application for her work at the university. I helped train her, but I refused to write any code because I wanted her to learn, and she did. Eventually the project got larger and she became a manager and hired someone whom she was to train to take over development. He took a look at the code, emailed her and said "Wow, this file looks really good, I assume your husband must have written this code?"

What a thing to say! Can you imagine how that must have felt? If I heard something like that said to me I'd want to curl up in a ball and die. And she had more experiences like this too, experiences she never had until she entered the technology space. And if you talk to women in this field, you'll hear these stories are common, and more: dismissal, harassment, rape threats if you become too visible or dare to speak out... not to mention there's the issue that most of the people in tech don't look like you, so you wonder if you really actually belonged, and you wonder if everyone else believes that too. Likewise with people of color, likewise with people in the LGBTQ space... stones and disrepair on the path, and sometimes you have to start a bit farther back.

To succeed at the race with privilege, of course you have to work hard. You have to train, you have to dedicate yourself, you have to run hard. This isn't meant to take away your accomplishments, or to say you didn't work hard. You did! It's to say that others have all these obstacles that we need to help clear from their path. No wonder so many give up the race. And there comes the responsibility and debt to be repaid: if you have privilege, put it to good work: pitch in. If you want that dream of everyone to have an equal place at the starting line to be true, help fix the track. But there's a lot of damage there... it's going to take a long time.

Morgan was knocking on the bathroom door. She wanted to know why I was crying when I was meant to be showering.

I was crying because my brain had played a cruel trick on me last night. It conjured a dream in which all the characters from the comic strip "Peanuts" represented myself and friends. Charlie Brown, the familiar but awkward everyman of the series, was absent from every scene, and in that way, heavily present.

I knew that Charlie Brown was absent because Charlie Brown had committed suicide.

I knew that Charlie Brown was my friend Matt Despears.

The familiar Peanuts imagery passed by: Linus (who was me), sat at the wall, but with nobody to talk to. Lucy held out the football, but nobody was there to kick it. Snoopy sat at his doghouse with an empty bowl, and nobody was there to greet him. And so on.

Then the characters, in silence, moved on with their lives, and the title scrolled by the screen... "We Miss You, Charlie Brown".

And so, that morning, I found myself in the shower, crying.

Why Peanuts? I don't know. I wouldn't describe myself as an overly energetic fan of the series. I also don't give too much credit for dream imagery as being necessarily important, since I think much tends to be the byproduct of the cleanup processes of the brain. But it hit home hard, probably because the imagery is so very familiar and repetitive, and so the absence of a key component amongst that familiarity stands out strongly. And maybe Charlie Brown just a good fit for Matt: awkward but loveable.

It has now been over six years since Matt has passed, and I find myself thinking of him often, usually when I have the urge to check in with him and remember that he isn't there. Before this recent move I was going through old drives and CDs and cleaning out and wiping out old junk, and found an archive of old chat logs from when I was a teenager. I found myself reliving old conversations, and most of it was utter trash... I felt embarrassed with my past self and nearly deleted the entire archive. But then I went through and read those chat logs with Matt. I can't say they were of any higher quality... my conversations with Matt seemed even more absurd on average than the rest. But I kept the chat logs. I didn't want to lose that history.

I felt compelled to write this up, and I don't entirely know why. I also nearly didn't write this up, because I think maybe this kind of writing can be dangerous. That may sound absurd, but I can speak from my experience of someone who frequently experiences suicidal ideation that the phrase "would anyone really miss me when I'm gone" comes to mind, and maybe this reinforces that.

I do think that society tends to romanticize depression and suicide in some strange ways, particularly this belief that suffering makes art greater. A friend of mine pointed this out to me for the first time in reference to John Toole's "A Confederacy of Dunces", often advertised and sold to others by, "and the author committed suicide before it was ever published!" But it would have been better to have more books by John Toole instead.

So as for "will anyone miss me if I'm gone", I want to answer that without romanticizing it. The answer is just "Yes, but it would be better if you were here."

A group of friends and I got together to play a board game recently. We sat around the table and had a good time. I drew a picture of "Batpope", one of Matt's favorite old jokes, and we left it on an empty spot at the table for Matt. But we would have rathered that Matt was there. His absence was felt. And that's usually how it is... like in the dream, we pass through the scenes of our lives, and we carry on, but there's a missing space, and one can feel the shape. There's no romance to that... just absence and memories.

We miss you, Matt Despears.

A couple of days ago I wrote about ActivityPub becoming a W3C Recommendation. This was one output of the Social Working Group, and the blogpost was about my experiences, and most of my experiences were on my direct work on ActivityPub. But the Social Working Group did more than ActivityPub; it also on the same day published WebSub, a useful piece of technology in its own right which amongst other things also plays a significant historical role in what is even ActivityPub's history (but is not used by ActivityPub itself), and it has also published several documents which are not compatible with ActivityPub at all, and appear to play the same role. This, to outsiders, may appear confusing, but there are reasons which I will go into in this post.

On that note, friend and Social Working Group co-participant Amy Guy just wrote a reasonably and (to my own feelings) highly empathizable frustrated blogpost (go ahead and read it before you finish this blogpost) about the kinds of comments you see with different members of different decentralized social web communities sniping at each other. Yes, reading the comments is always a precarious idea, particularly on tech news sites. But what's especially frustrating is seeing comments that we either:

• Ignored prior art, and just went off and wrote ActivityPub and other specifications with no attention to what was done (the comment "including prior art by its creators" is especially strange, because I assume this means Evan Prodromou (co-chair of the SocialWG) ignoring OStatus, which he largely wrote... I'm not sure how we could have ignored the prior art, in that case, if we were trying. Evan did encourage the move from anything using OStatus to using ActivityPub, but it certainly wasn't out of ignorance for his own work.)
• Pushed out multiple conflicting standards as if on a whim

These comments seem to be being made by people who were not part of the standards process, so as someone who spent three years of their life on it, let me give the perspective of someone who was actually there.

So yes, first of all, it's true that in the end we pushed out two "stacks" that were mostly incompatible. These would more or less be the "restful + linked data" stack, which is ActivityPub and Linked Data Notifications using ActivityStreams as its core (but extensible) vocabulary (which are directly interoperable, and use the same "inbox" property for delivery), and the "Indieweb stack", which is Micropub and Webmention. (And there's also WebSub, which is not really either specifically part of one or the other of those "stacks" but which can be used with either, and is of such historical significance to federation that we wanted it to be standardized.) Amy Guy did a good job of mapping the landscape in her Social Web Protocols document.

Gosh, two stacks! It does kind of look confusing, if you weren't in the group, to see how this could have happened. Going through meeting logs is boring (though the meeting logs are up there if you feel like it) so here's what happened, as I remember it.

First of all, we didn't just start out with two stacks, we started out with three. At the beginning we had the linked data folks, the RESTful "just speak plain JSON" development type folks, and the Indieweb folks. Nobody really saw eye to eye at first, but eventually we managed to reach some convergence (though not as much as I would have liked). In fact we managed to merge two approaches entirely: ActivityPub is a RESTful API that can be read and interpreted as just JSON, but thanks to JSON-LD you have the power of linked data for extensions or maybe because you really like doing fancy RDF the-web-is-a-graph things. And ActivityPub uses the very same inbox of Linked Data Notifications, and is directly interoperable. Things did not start out as directly interoperable, but Sarven Capadisli and Amy Guy (who was not yet a co-author of ActivityPub) were willing to sit down and discuss and work out the details, and eventually we got there.

Merging the RESTful + Linked Data stuff with the Indieweb stuff was a bit more of a challenge, but for a while it looked like even that might completely happen. For those that don't know, Linked Data type people and Indieweb type people have, for whatever reason, historically been at each others' throats despite (or perhaps because of) the enormous similarity between the kind of work that they're doing (the main disagreements being "should we treat everything like a graph" and "are namespaces a good idea" and also, let's be honest, just historical grudges). But Amy Guy long made the case in the group that actually the divisions between the groups were very shallow and that with just a few tweaks we could actually bridge the gap (this was the real origin of the Social Web Protocols document, which though it eventually became a document of the different things we produced, was originally an analysis of how they weren't so different at all). At the face to face summit in Paris (which I did not attend, but ActivityPub co-editor Jessica Tallon did) there was apparently an energetic meeting over a meal where I'm told that Jessica Tallon and Aaron Parecki (editor of Micropub and Webmention) hit some kind of epiphany and realized yes, by god, we can actually merge these approaches together. Attending remotely, I wasn't there for the meal, but when everyone returned it was apparent that something had changed: the conversation had shifted towards reconciling differences. Between the Paris face to face meeting and the next one, energy was high and discussions active on how to bring things together. Aaron even began to consider that maybe Micropub (and/or? I forget if it was just one) Webmention could support ActivityStreams, since ActivityStreams already had an extension mechanism worked out. At the next face to face meeting, things started out optimistic as well... and then suddenly, within the span of minutes, the whole idea of merging the specs fell apart. In fact it happened so quickly that I'm not even entirely sure what did it, but I think it was over two things: one, Micropub handled an update of fields where you could add or remove a specific element from a list (without giving the entire changed list as a replacement value) and it wasn't obvious how it could be done with ActivityPub, and two, something like "well we already have a whole vocabulary in Microformats anyway, we might as well stick with it." (I could have the details wrong here a bit... again, it happened very fast, and I remember in the next break trying to figure out whether or not things did just fall apart or not.)

With the the dream of Linked Data and Indieweb stuff being reconciled given up on, we decided that at least we could move forward in parallel without clobbering, and in fact while actively supporting, each other. I think, at this point, this was actually the best decision possible, and in a sense it was even very fruitful. At this point, not trying to reconcile and compromise on a single spec, the authors and editors of the differing specifications still spent much time collaborating as the specifications moved forward. Aaron and other Indieweb folks provided plenty of useful feedback for ActivityPub and the ActivityPub folks provided plenty of useful feedback for the Indieweb folks, and I'd say all our specifications were improved greatly by this "friendly treaty" of sorts. If we could not unify, we could at least cooperate, and we did.

I'd even say that we came to a good amount of mutual understanding and respect between these groups within the Social Web Working Group. People approached these decentralization challenges with different building blocks, assumptions, principles, and goals... hence at some point they've encountered approaches that didn't quite jive with their "world view" on how to do it right (TM). And that's okay! Even there, we have plenty of space for cooperation and can learn from each other.

This is also true with the continuation of the Social Web Working Group, which is the SocialCG, where the two co-chairs are myself and Aaron Parecki, who are both editors of specifications of the conflicting "stacks". Within the Social Web Community Group we have a philosophy that our scope is to work on collaboration on social web protocols. If you use a different protocol than another person, you probably can still collaborate a lot, because there's a lot of overlap between the problem domains between social web protocols. Outside the SocialWG and SocialCG it still seems to be a different story, and sadly linked data people and Indieweb people seem to still show up on each others' threads to go after each other. I consider that a disappointment... I wish the external world would reflect the kind of sense of mutual understanding we got in the SocialWG and SocialCG.

Speaking of best attempts at bringing unity, my main goal at participating in the SocialWG, and my entire purpose of showing up in the first place, was always to bring unity. The first task I performed over the course of the first few months at the Social Working Group was to try to bring all of the existing distributed social networks to participate in the SocialWG calls. Even at that time, I was worried about the situation with a "fractured federation"... MediaGoblin was about to implement its own federation code, and I was unhappy that we had a bunch of libre distributed social network projects but none of them could talk to each other, and no matter what we chose we would just end up contributing to the problem. I was called out as naive (which I suppose, in retrospect, was accurate) for a belief that if we could just get everyone around the table we could reconcile our differences, agree on a standard that everyone could share in, and maybe we'd start singing Kumbaya or something. And yes, I was naive, but I did reach out to everyone I could think of (if I missed you somehow, I'm sorry): Diaspora, GNU Social, Pump.io (well, they were already there), Hubzilla, Friendica, Owncloud (later Nextcloud)... etc etc (Mastodon and some others didn't even exist at this point, though we would connect later)... I figured this was our one chance to finally get everyone on board and collaborate. We did have Diaspora and Owncloud participants for a time (and Nextcloud even has begun implementing ActivityPub), and plenty of groups said they'd like to participate, but the main barrier was that the standards process took a lot of time (true story), which not everyone was able to allocate. But we did our best to incorporate and respond to feedback whever we got it. We did detailed analysis on what the major social networks were providing and what we needed to cover as a result. What I'm trying to say is: ActivityPub was my best attempt to bring unity to this space. It grew out of direct experiences from developing previous standards between OStatus, the Pump API, and over a decade of developing social network protocols and software, including by people who pioneered much of the work in that territory. We tried through long and open comment periods to reconcile the needs of various groups and potential users. Maybe we didn't always succeed... but we did try, and always gave it our best. Maybe ActivityPub will succeed in that role or maybe it won't... I'm hopeful, but time is the true test.

Speaking of attempting to bring unity to the different decentralized social network projects, probably the main thing that disappoints me is the amount of strife we have between these different projects. For example, there are various threads pitting Mastodon vs GNU Social. In fact, Mastodon's lead developer and GNU Social's lead developer get along just fine... it's various members of the communities of each that tend to (sounds familiar?) be hostile.

Here's something interesting: decentralized social web initiatives haven't yet faced an all-out attack from what would be presumably be their natural enemies in the centralized social web: Facebook, Twitter, et all. I mean, there have been some aggressions, in the senses that bridging projects that let users mirror their timelines get shut down as terms of service violations and some comparatively minor things, but I don't know of (as of yet) an outright attack. But maybe they don't have to: participants in the decentralized social web is so good at fighting each other that apparently we do that work for them.

But it doesn't have to be that way. You might be able to come to consensus on a good way forward. And if you can't come to consensus, you can at least have friendly and cooperative communication.

And if somehow, you can't do any of that, you just not openly attack each other. We've got enough hard work to fight to make the federated social web work without fighting ourselves. Thanks.

Update: A previous version of this article said "I even saw someone tried to write a federation history and characterize it as war", but it's been pointed out that I'm being unfair here, since the very article I'm pointing to itself refutes the idea of this being war. Fair point, and I've removed that bit.

Having spent the majority of the last three years of my life on it, I'm happy to announce that ActivityPub is now a W3C Recommendation. Whew! At last! Horray! Finally! I've written some more words on this over on the FSF's blog so maybe read that.

As for things I didn't put there, that fit more on a personal blog? I guess that's where I speak about my personal life experience and feelings about it and I would say they're a mix of elation (for making it), relief (also for making it, because it wasn't always clear that we would), and burnout (I had no idea this process was going to suck up so much of my life).

I didn't expect this to take over my life so thoroughly. I did say this bit on the FSF blogpost but when Jessica Tallon and I got involved in the Social Working Group we figured we were just showing up for an hour a week to make sure things were on track. I did think the goal of the Social Working Group was the right one: we had a lot of libre social networks but they were largely fractured and failed at interoperability... surely we could do better if we got everyone in a room together! (Getting everyone in the room wasn't easy and didn't always happen, though I sure as heck tried, particularly early on.) But I figured the other people in the room would be the experts, the responsible ones, and we'd just be tagging along to make sure our needs were met. Well, the next thing you know we're co-editors of ActivityPub, and that time grew from an hour a week to filling most of my week to sometimes urgent, grueling deadlines (granted, I made most of them a lot more complicated than I needed to be by doing example implementations in obscure languages, etc etc).

I'm feeling great about things now, but that wasn't always the case through this. I've come to learn how hard standards work is, and I've been doing other specification work recently too (more on that in a coming blogpost), but I'll say that for whatever reason (and I can think of quite a few, but it's not worth going into here), ActivityPub has been far harder than anything else I've worked on in the standards space. (Maybe that's just because it's the first standard I've gotten to completion though.)

In fact, in early-to-middle 2017 I was in quite a bit of despair, because it seemed clear that ActivityPub was going to not make it in time as an official recommended standard. The Social Working Group's charter was going to run out at mid-2017, and it had already been extended once... apparently getting a second extension was nearly unheard of. I resigned myself to the idea that ActivityPub would be published as a note, but that there was no way that we would be able to make it to getting the shiny foil stamp of being an actual recommended standard. Instead, I shifted my effort to making sure that my ActivityPub implementation work would support enough of ActivityStreams (which is what ActivityPub uses as its vocabulary) to make sure that at least that would make it as a standard with all the components we required, since we at least needed to be able to refer to that vocabulary.

But Mastodon saved ActivityPub. I'll admit that at first I was skeptical about all the hype I was hearing about Mastodon... but Amy Guy (co-author of ActivityPub, and whose PHD thesis, "Presentation of Self on a Decentralised Web", is worth a read at the memorable domain of dr.amy.gy) convinced me that I really ought to check out what was going on in Mastodon land. And I found I really did like what was happening there... and connected to a community that felt like what I had missed from the heyday of StatusNet/identi.ca, while having a bit of its own flavor of culture, one that I really felt at home in. It turned out this was good timing... Mastodon was having trouble expanding the privacy needs of its users on OStatus, and it turns out private addressing was exactly one of the reasons that ActivityPub was developed. (I'm not claiming credit for this, I'm just talking from my perspective... the Mastodon ActivityPub implementation issue can give you a better sense of where credit is due, and here I didn't really do much.) This interest came right at the right time... it began to also drum up interest from many other participants too... and it pretty much directly lead to another extension to the Social Working Group, giving us until the end of 2017 to wrap up the work on standardizing ActivityPub. Whew!

But Mastodon is not alone. Today there are a growing number of implementers of ActivityPub. I'd encourage you, if you haven't, to watch this video of PeerTube and Mastodon federating over ActivityPub. Pretty cool stuff! ActivityPub has been a massive group effort, and I'm relieved to see that all that hard work has paid off, for all of us.

Meanwhile, there's a lot to do still ahead. MediaGoblin, ironically, has fallen behind on its own federation support in the interest of advancing federation standards (we have some federation code, but it's for the old pre-ActivityPub Pump API, and it's bitrotted quite a bit) and I need to figure out what the next steps are and discuss with the community (expect more on that in the next few months, and sure to be discussed at my talk at Libreplanet 2018). And ActivityPub may be "done" in the sense that "it made it through the standards process", but some of the most interesting work is still ahead. The Social Web Community Group, of which I am co-chair, meets bi-weekly to talk and collaborate on the interesting problems that implementers of libre networks are encountering. (It's open to everyone, maybe you should join?)

On that note, in a recent Social Web Community Group meeting, Evan Prodromou was showing off some of his latest ActivityPub projects (tags.pub and places.pub). I'm paraphrasing here, but he said something interesting, which has stuck with me: "We did all that standardizing work, and that's great, but now we get to the fun part... now we get to build things."

I agree. I look forward to what the next few years of fun ActivityPub development bring. Onwards!