Archives

Tags

Posts with tag "foss"

Racket is an acceptable Python

By Christopher Lemmer Webber on Tue 09 July 2019

A little over a decade ago, there were some popular blogposts about whether Ruby was an acceptable Lisp or whether even Lisp was an acceptable Lisp. Peter Norvig was also writing at the time introducing Python to Lisp programmers. Lisp, those in the know knew, was the right thing to strive for, and yet seemed unattainable for anything aimed for production since the AI Winter shattered Lisp's popularity in the 80s/early 90s. If you can't get Lisp, what's closest thing you can get?

This was around the time I was starting to program; I had spent some time configuring my editor with Emacs Lisp and loved every moment I got to do it; I read some Lisp books and longed for more. And yet when I tried to "get things done" in the language, I just couldn't make as much headway as I could with my preferred language for practical projects at the time: Python.

Python was great... mostly. It was easy to read, it was easy to write, it was easy-ish to teach to newcomers. (Python's intro material is better than most, but my spouse has talked before about some major pitfalls that the Python documentation has which make getting started unnecessarily hard. You can hear her talk about that at this talk we co-presented on at last year's RacketCon.) I ran a large free software project on a Python codebase, and it was easy to get new contributors; the barrier to entry to becoming a programmer with Python was low. I consider that to be a feature, and it certainly helped me bootstrap my career.

Most importantly of all though, Python was easy to pick up and run with because no matter what you wanted to do, either the tools came built in or the Python ecosystem had enough of the pieces nearby that building what you wanted was usually fairly trivial.

But Python has its limitations, and I always longed for a lisp. For a brief time, I thought I could get there by contributing to the Hy project, which was a lisp that transformed itself into the Python AST. "Why write Python in a syntax that's easy to read when you could add a bunch of parentheses to it instead?" I would joke when I talked about it. Believe it or not though, I do consider lisps easier to read, once you are comfortable to understand their syntax. I certainly find them easier to write and modify. And I longed for the metaprogramming aspects of Lisp.

Alas, Hy didn't really reach my dream. That macro expansion made debugging a nightmare as Hy would lose track of where the line numbers are; it wasn't until that when I really realized that without line numbers, you're just lost in terms of debugging in Python-land. That and Python didn't really have the right primitives; immutable datastructures for whatever reason never became first class, meaning that functional programming was hard, "cons" didn't really exist (actually this doesn't matter as much as people might think), recursive programming isn't really as possible without tail call elimination, etc etc etc.

But I missed parentheses. I longed for parentheses. I dreamed in parentheses. I'm not kidding, the only dreams I've ever had in code were in lisp, and it's happened multiple times, programs unfolding before me. The structure of lisp makes the flow of code so clear, and there's simply nothing like the comfort of developing in front of a lisp REPL.

Yet to choose to use a lisp seemed to mean opening myself up to eternal yak-shaving of developing packages that were already available on the Python Package Index or limiting my development community an elite group of Emacs users. When I was in Python, I longed for the beauty of a Lisp; when I was in a Lisp, I longed for the ease of Python.

All this changed when I discovered Racket:

  • Racket comes with a full-featured editor named DrRacket built-in that's damn nice to use. It has all the features that make lisp hacking comfortable previously mostly only to Emacs users: parenthesis balancing, comfortable REPL integration, etc etc. But if you want to use Emacs, you can use racket-mode. Win-win.
  • Racket has intentionally been built as an educational language, not unlike Python. One of the core audiences of Racket is middle schoolers, and it even comes with a built-in game engine for kids. (The How to Design Programs prologue might give you an introductory taste, and Realm of Racket is a good book all about learning to program by building Racket games.)
  • My spouse and I even taught classes about how to learn to program for humanities academics using Racket. We found the age-old belief that "lisp syntax is just too hard" is simply false; the main thing that most people lack is decent lisp-friendly tooling with a low barrier to entry, and DrRacket provides that. The only people who were afraid of the parentheses turned out to be people who already knew how to program. Those who didn't even praised the syntax for its clarity and the way the editor could help show you when you made a syntax error (DrRacket is very good at that). "Lisp is too hard to learn" is a lie; if middle schoolers can learn it, so can more seasoned programmers.
  • Racket might even be more batteries included than Python. At least all the batteries that come included are generally nicer; Racket's GUI library is the only time I've ever had fun in my life writing GUI programs (and they're cross platform too). Constructing pictures with its pict library is a delight. Plotting graphs with plot is an incredible experience. Writing documentation with Scribble is the best non-org-mode experience I've ever had, but has the advantage over org-mode in that your document is just inverted code. I could go on. And these are just some packages bundled with Racket; the Package repository contains much more.
  • Racket's documentation is, in my experience, unparalleled. The Racket Guide walks you through all the key concepts, and the Racket Reference has everything else you need.
  • The tutorials are also wonderful; the introductory tutorial gets your feet wet not through composing numbers or strings but by building up pictures. Want to learn more? The next two tutorials show you how to build web applications and then build your own web server.
  • Like Python, even though Racket has its roots in education, it is more than ready for serious practical use. These days, when I want to build something and get it done quickly and efficiently, I reach for Racket first.

Racket is a great Lisp, but it's also an acceptable Python. Sometimes you really can have it all.

Libre Lounge

By Christopher Lemmer Webber on Fri 08 February 2019

Did I somehow not blog here that I started co-hosting a podcast named Libre Lounge with my friend Serge Wroclawski? We're talking about all sorts of topics facing user freedom. Take a look at the archive and you might find something you like!

At the time of writing we've also had on one guest, Karen Sandler. This was a really nice treat since the show Free as in Freedom, which Karen co-hosts, is clearly a big influence on Libre Lounge. Anyway, we plan to have on lots more guests in the future.

Hope you enjoy!

I've been awarded the Samsung Stack Zero Grant

By Christopher Lemmer Webber on Thu 31 January 2019

Good news everyone! I've been awarded the Samsung Stack Zero Grant. But why not quote them?

Christopher Lemmer Webber is the co-editor and co-author of the now-ubiquitous ActivityPub protocol. While it provides a great framework for creating, updating, and deleting content across applications, it doesn’t provide any standardised mechanism for secure authorisation. With Spritely, Webber will work on extending the protocol in a backward-compatible manner, while at the same time building tools and applications that showcase its use. This will enable developers to build applications that enable richer interactions through a federated standard.

This should fund my next couple of years of work on full time advancement of the fediverse.

You may remember that I've talked about Spritely before. In fact I am finally in launch-mode... I am currently sitting in a wizard's tower at a hackathon, getting out the first release of Golem, a Spritely artifact.

Anyway, I'll be at FOSDEM 2019 giving a talk and on a panel. And after that I'll be speaking at CopyleftConf. Maybe I'll see you?

More news soon...

Announcing Goblins (the actor model foundation of Spritely)

By Christopher Lemmer Webber on Wed 31 October 2018

In my last post I talked about launching the Spritely project in all its ambitiousness. As I said, I plan on releasing it as a series of "demos". What I mean by "demo" is fairly loose: things you can try and see, artifacts to indicate progress as I go. Convergence slowly towards a goal.

Over the last week I released the first version of the foundation of this work, which I'm calling Goblins. v0.1, a pre-pre-alpha release, is now out the door. Goblins is an actor model implementation for the Racket programming language. I think if you know some Racket, then the Goblins documentation should be fairly approachable, and I'd love to hear feedback. Not everything is documented, but it should give you a nice taste of what the core ideas are. (Astute readers may observe that Goblins' name shares striking resemblance to another project I have worked on; this is no coincidence and is indeed a hint as to what I think the future direction of that area of work is.)

Most live distributed systems are in some way a language-level actor model implementation, but knowing that you're an actor model implementation may change how you do things. (ActivityPub is itself an actor model protocol, though I think not all implementors realize that.) Goblins is the third generation of an actor model I've written, the previous ones being XUDD and 8sync. (Maybe if you count the big rewrite I did of 8sync to run it on top of fibers, it could be the fourth generation.)

If you read the Spritely blogpost you'll know that distributed games are part of the long con here. This isn't the first time I've written a game system on top of one of these actor model implementations; in fact if you scroll down on 8sync website you'll (currently) see a video of me presenting on Mudsync, a multi-user dungeon written in 8sync, where in the talk instead of slides I had rooms, and the audience was moving around in the world and I was changing it in response to their suggestions.

But the drive of the actor model connected to social distributed games goes back further for me. I initially got interested in an actor model the first time I tried to work on the whole multiplayer game front about a decade ago. A friend and I spent a couple hours experimenting with threads and locks and after the first demo it was obvious that was never going to work. I remembered I had been told by a former coworker that they had used a system that had an actor model + coroutines as its basis; I didn't know what either of those things were yet, but as I looked into them it became clear to me that this was the right route. But one piece since then until recently remained a mystery to me, which was how to manage the complex web of authorization between the objects.

Last year at TPAC, the World Wide Web consortium's conference, I was staying with some people who asked me what I would do if I had unlimited time to work on whatever I wanted. I sheepishly admitted to my dream of a federated social web as a distributed game, and expected to be laughed out of the room. I was surprised to find out that nearly everyone I was working with had some background in this same work and dream. This gave me some hope that I wasn't completely foolish, and the observation that even failure in persuing this ambitious dream would probably result in useful artifacts.

And last year at Rebooting Web of Trust, I hit a big breakthrough. I had the chance to meet Mark S. Miller in person, someone who's work I read but barely understood at the time, and this meeting in some ways changed the direction of my career. For one thing, it lead to us collaborating on the foundation for the ocap-ld specification, which came out of a paper we collaborated on from that event (also available in nicely formatted PDF form). But what I also discovered was that Mark Miller and company had actually built the distributed and secure social network dream in the late 1990s and called it Electric Communities (rare video of it being demonstrated, and a rare video of some of the out-there ideas). What I found out was that they had solved most of the remaining questions about authorization and security through object capabilities, and that was through a pile of object capability patterns built on top of the actor model. While Electric Communities did not survive, its foundation of the the "E" programming language lived on, pulling in many of its good ideas with it. I proceeded to read as much documentation about E and surrounding capability patterns as I could, and this has majorly influenced Goblins' design. (Don't underestimate the depth of ideas on erights.org just because it looks like an ancient Geocities page... it did come from that era after all...)

Another key insight came from discovering the Object-Capability Security in Virtual Environments paper. I won't go into details on this except to say that it's a must-read for this problem domain, with the caveat that it pulls in a lot of terminology baggage from the object capability (aka ocap) community. The ocap community is one of those groups that has so many of the right ideas but which unfortunately is also drowning in its own verbiage, despite true efforts to make the ideas more accessible. We (and I guess I'm part of this group now, so "we" is appropriate) need to do better, but it isn't easy.

Goblins is still in its very early days. Its api is unstable and it surely has bugs (this one in particular I need to deal with, and which even exposed an underlying bug in Racket, which did get fixed quickly because the Racket community is amazing). Please don't build anything production-oriented on top of it yet. That said, I'd love users to try things. Check out the docs, see if they're up your alley.

If you like this kind of work and want to see more of it, consider donating; I don't have a corporate sponsor, so progress is funded by people like you. And keep watching here; more Spritely demos/artifacts/releases on the horizon.

Spritely: towards secure social spaces as virtual worlds

By Christopher Lemmer Webber on Sun 14 October 2018

If you follow me on the fediverse, maybe you already know. I've sent an announcement to my work that I am switching to doing a project named Spritely on my own full time. (Actually I'm still going to be doing some contracting with my old job, so I'll still have some income, but I'll be putting a full 40 hours a week into Spritely.)

tl;dr: I'm working on building the next generation of the fediverse as a distributed game. You can support this work if you so wish.

What on earth is Spritely?

"Well, vaporware currently", has been my joke since announcing it, but the plans, and even some core components, are starting to congeal, and I have decided it's time to throw myself fully into it.

But I still haven't answered the question, so I'll try to do so in bullet points. Spritely:

  • Aims to bring stronger user security, better anti-abuse tooling, stronger resistance against censorship, and more interesting interactions to users of the fediverse.
  • Is based on the massively popular ActivityPub standard (which I co-authored, so I do know a thing or two about this).
  • Aims to transform distributed social networks into distributed social games / virtual worlds. The dreams of the 90s are alive in Spritely.
  • Recognizes that ActivityPub is based on the actor model, and a pure version of the actor model is itself already a secure object capability system, so we don't have to break the spec to gain those powers... just change the discipline of how we use it.
  • Will be written in Racket.
  • Is an umbrella project for a number of modular tools necessary to get to this goal. The first, an object capability actor model system for Racket named Goblins, should see its first public release in the next week or two.
  • And of course it will be 100% free/libre/open source software.

That's a lot to unpack, and it also may sound overly ambitious. The game part in particular may sound strange, but I'll defend it on three fronts. First, not too many people run federated social web servers, but a lot of people run Minecraft servers... lots of teenagers run Minecraft servers... and it's not because Minecraft has the best graphics or the best fighting (it certainly doesn't), it's because Minecraft allows you to build a world together with your friends. Second, players of old MUDs, MOOs, MUSHes and etc from the 90s may recognize that modern social networks are structurally degenerate forms of the kinds of environments that existed then, but contemporary social networks lack the concept of a sense of place and interaction. Third, many interesting projects (Python's Twisted library, Flickr, much of object capability security patterns) have come out of trying to build such massively multiplayer world systems. Because of this last one in particular, I think that shooting for the stars means that if we don't make it we're likely to at least make the moon, so failure is okay if it means other things come out of it. (Also, four: it's a fun and motivating use case for me which I have explored before.)

To keep Spritely from being total vaporware, the way I will approach the project is by regularly releasing a series of "demos", some of which may be disjoint, but will hopefully increasingly converge on the vision. Consider Spritely a skunkworks-in-the-public-interest for the federated social web.

But why?

Standardizing ActivityPub was a much more difficult effort than anticipated, but equally or more so more successful than I expected (partly due to Mastodon's adoption launching it past the sound barrier). In that sense this is great news. We now have dozens of projects adopting it, and the network has (at last I looked) over 1.5 million registered users (which isn't the same as active users).

So, mission accomplished, right? Well, there are a few things that bother me.

  • The kind of rich interactions one can do are limited by a lack of authorization policy. Again, I believe object capabilities provide this, but it's not well explained to the public how to use it. (By contrast, Access Control Lists and friends are absolutely the wrong approach.)
  • Users are currently insufficiently protected from spam, abuse, and harassment while at the same time administrators are overwhelmed. This is leading a number of servers to move to a whitelisting of servers, which both re-centralizes the system and prioritizes big instances over smaller instances (it shouldn't matter what instance size you're on; arguably we should be encouraging smaller ones even). There are some paths forward, and I will hint at just one: what would happen if instead of one inbox, we had multiple inboxes? If I don't know you, you can access me via my public inbox, but maybe that's heavily moderated or you have to pay "postage". If I do know you, you might have an address with more direct access to me.
  • Relatedly, contemporary fediverse interfaces borrow from surveillance-capitalism based popular social networks by focusing on breadth of relationships rather than depth. Ever notice how the first thing Twitter shows you when you hover over a person's face is how many followers they have? I don't know about you, but I immediately compare that to my own follower count, and I don't even want to. This encourages high school popularity contest type bullshit, and it's by design. What if instead of focusing on how many people we can connect to we instead focused on the depth of our relationships? Much of the fediverse has imported "what works" directly from Facebook and Twitter, but I'd argue there's a lot we can do if we drop the assumption that this is the ideal starting base.
  • The contemporary view in the fediverse is that social scoping is like Python scoping: locals (instance) and globals (federation). Instance administrators are even encouraged to set up to run communities based on a specific niche, which is a nice reason to motivate administrators but it causes problems: even small differences between servers' expected policies often result in servers banning each other entirely. (Sometimes this is warranted, and I'm not opposed to moderation but rather looking for more effective forms of it.) Yet most of us are one person but part of many different communities with different needs. For instance, Alice may be a computer programmer, a tabletop game enthusiast, a fanfiction author, and a member of her family. In each of those settings she may present herself differently and also have different expectations of what is acceptable behavior. Alice should not need multiple accounts for this on different servers, so it would seem the right answer for community gathering is closer to something like mailing lists. What is acceptable at the gaming table may not be acceptable at work, and what happens on the fanfiction community perhaps does not need to be shared with one's family, and each community should be empowered to moderate appropriately.
  • I'd like to bridge the gap between peer to peer and federated systems. One hint as to how to do this: what happens when you run ActivityPub servers over Tor onion services or I2P? What if instead of our messages living at http addresses that could down, they could be securely addressed by their encrypted contents?
  • Finally, I will admit the most urgent reason for these concerns... I'm very concerned politically about the state of the world and what I see as increasing authoritarianism and flagrant violations of human rights. I have a lot of worry that if we don't normalize use of decentralized and secure private systems, we will lose the ability to host them, though we've never needed them more urgently.

There are a lot of opportunities, and a lot of things I am excited about, but I am also afraid of inaction and how many regrets I will have if I don't try. I have the knowledge, the privilege, and the experience to at least attempt to make a dent in some of these things. I might not succeed. But I should try.

Who's going to pay for all this?

I don't really have a funding plan, so I guess this is kind of a non-answer. However, I do have a Patreon account you could donate to.

But should you donate? Well, I dunno, I feel like that's your call. Certainly many people are in worse positions than I am; I have a buffer and I still am doing some contracting to keep myself going for a while. Maybe you know people who need the money more than I do, or maybe you need it yourself. If this is the case, don't hesitate: take care of yourself and your loved ones first.

That said, FOSS in general has the property of being a public good but tends to have a free rider problem. While we did some fundraising for some of this stuff a few years ago, I gave the majority of the money to other people. Since then I've been mostly funding work on the federated social web myself in one way or another, usually by contracting on unrelated or quasi-related things to keep myself above the burn rate. I have the privilege and ability to do it, and I believe it's critical work. But I'd love to be able to work on this with focus, and maybe get things to the point to pull in and pay other people to help again. Perhaps if we reach that point I'll look at putting this work under a nonprofit. I do know I'm unwilling to break my FOSS principles to make it happen.

Anyway... you may even still be skeptical after reading all this about whether or not I can do it. I don't blame you... even I'm skeptical. But I'll try to convince you the way I'm going to convince myself: by pushing out demos until we reach something real.

Onwards and upwards!