Update: Well, you post a thing, and sometimes that's enough for people to come and help you realize how wrong you are. Which is good! There are a number of ways forward (some obvious in retrospect). For one, pyrsistent does exist and looks nice and... well it's even actively developed. But even aside from that, there are several clean solutions: wrapper objects which "lock" the child object with getters but no setters, or even just using alist style tuples of tuples for a fake hashmap. Options are indeed abound.

And the exception thing? Well, that wasn't listed as a permanent problem below, but the solution is even easier! It would be simple to have a MessageError("some_identifier") which has a minimalist identifier which can be passed across the wire, and the directive of this error can be a special case.

Anyway, you can read the original post below. But it's good to be wrong. XUDD no longer has reason to be dead. Long live XUDD!

tl;dr: Kind of along post, but basically the lack of good functional data structures in Python has kinda killed the project.

One of my favorite projects ever has been to work on XUDD, an asynchronous actor model system for Python. Originally born out of a quest to build a MUD (hence the name), but eventually became the focus of being an actor model itself, it was a really interesting exploration for me.

There's a lot of things I like about the actor model... for one thing, functional programming is all the rage, right? But not all systems are easy to express in a purely functional style, and done right an actor model can be fairly object oriented'y, but done right, you can have your mutable cake and eat it too, safely! Your actor can mutate some of its own variables, but when it communicates across the wire, since it's a "shared nothing" environment you can get even better scale-to-the-moon type functionality than in many functional language systems: it's trivial to write code where actors communicate across multiple processes or machines in just the same way as if they were all on the same machine in the same process and thread.

I put a lot of thought into XUDD, and I've looked into some of the alternatives like Pykka, and I still think XUDD has some ideas that kicks butt on other systems. I still think the use of coroutines feels very clean and easy to read, the "hive" model is pretty nice, and the way it's built on top of the awesome asyncio system for Python are all things I'm happy with.

So, every once in a while I get an email from someone who reads the XUDD documentation and also gets excited and asks me what's going on.

The sad reality is: I'm stuck. I'm stuck on two fronts, and one I can figure my way out of, but the other one doesn't seem easy to deal with in Python as-is.

The first issue is of error propagation. This is solvable, but when an exception is raised, it would be nice to propagate this back to the original actor. There are some side issues I'm not sure about: in an inter-hive-communication (read: multiple machines or processes) type scenario, should we use standard exceptions and try to import and reproduce the same exception that was raised elsewhere? That seems like it could be... gnarly to do. Raising the error inside the original routine is also a bit tricky, but not too hard; python's coroutines can support it and I just need to think about it. So exceptions are annoying but solvable.

But the other issue... I'm not sure what to do about it. Basically, it's an issue of a lack of immutable types, or we might even say "purely functional datastructures" that are robust enough to continue with. Why does that matter? Messages sent between actors shouldn't have any mutable data. It's fine and well and even a nice feature for actors to be able to have mutable data within themselves, and actually even provide a nice way to pull of things that are just damned hard in purely functional systems, but between actors, mutable data is a no-no.

It's easy to see why: say we have a function that has a list in it, say of a number of children in my classroom, and I send this list over to an actor that controls some sort of database, right? I'm doing things in a nice, fancy coroutine'ish type way, which means my function can just suspend mid-execution while it waits for that database actor to generate some sort of reply and send it back to me. What happens if that other function pops one of the items off the list, or appends to it, or in some way actually mutates the list? Now, when my function continues, it'll be operating on a differently formed list than the one it thinks it has. I might have a reference to the third item in the list, but it turns out that there isn't a third item in the list anymore, because the other function popped it off. This can introduce all sorts of subtle bugs, and it's bumming me out that I don't have a good solution to them.

Now, there's a way around this: you can serialize each and every message completely before sending it to another actor. And of course, if actors are on different threads, processes, or even entirely different machines, of course we'd do this. But XUDD has the concept of actors being on the same hive, and there are a number of reasons for this, but one of them is that for local message passing, packing and unpacking data in some sort of serialized format for every call slows things down by a lot. When I originally began designing XUDD, the plan was for games that might need to shard out to a number of different servers but have players that can traverse different parts of the system and communicate with other shards (without knowing or the code mostly knowing that it's communicating with other actors that are technically remote). I want to be able to pass many messages at once to actors that are on the same hive, while still having a totally safe time of doing so. But there's no way to do so without a nice set of immutable / "purely functional" types, and Python just doesn't have this right now. None of the third party libraries I've found seem well maintained (am I missing something?), and the standard library is fairly deficient here. Why? I'm not really sure. I guess Python's history is just synchronously imperative enough that it just hasn't mattered.

I'd like to continue research into the actor model... I have some projects I'd like to work on where the actor model seems perfectly tuned to those tasks. What to do?

Well, I'm not really sure... I guess I could just serialize everything all the time, but it's kind of a bummer to me that so many cycles would be wasted for local computation. Maybe it's a dumb reason to feel exhausted with things, but that's the state of it. I'm not enough of a datastructure wizard to implement these things myself, but they exist. I've thought about giving up on XUDD being a Python project and to move over to something else... Guile has a cooperative REPL which would be great for debugging, and I really like the community there, so maybe that would be a nice place to go. Not really sure there's anything else I'm interested enough in at the moment. I think I'd miss Python. Or maybe I'm over-thinking everything in the first place? (Wouldn't be the first time.)

Maybe there's another way out. If you have any ideas, contact me.

While working on MediaGoblin's upcoming release, and (dare I say) feeling a bit bored and anxious and frustrated about the whole process, I decided that maybe it was time to look into some sort of entertainment. But for the most part, I feel like I've played all the interesting free software games out there. I've mostly beaten even the games of my old proprietary console using past. What to do?

I wondered what the state of good ol fashioned MUD systems are. I've had dreams for years of building such a system from scratch (hence my interest in the actor model). Answer: they feel like they're in the same place they were around 2000. But maybe this isn't bad? I tried playing some dungeon crawling'y MUDs but found myself a bit bored. But having remembered that Rob Myers has talked plenty about LambdaMOO, so that seems like good enough reason to connect and look around. (Note, connecting is not hard! I am using Mudlet at the moment.)

So, in-between actual coding and fretting about this and that, I've been getting in some downtime, poking around the LambdaMOO environment. And at first I was fairly unimpressed... this thing is what, 20 rooms big? What can I do here? But I've been finding some interesting things:

• That LambdaMOO is both the software and one particular server is fairly confusing, but no matter: both are of interesting construction. They're both also very old, 25 years old at this point (egads), but particularly as to the world, that leads to much depth.
• The main rooms of the house (see [help geography]{.title-ref}) are not as interesting as the places they lead to. You can enter a model railroad and actually walk around that imaginary city and end up in some interesting places, reading graffiti off of nightclub walls, etc. Find an old teletype and type on it, or play one of the functioning (!) board games sitting around. A dusty hallway leads to a full hack and slash style game engine, coded live inside the system itself.
• Most of the world is not really a dungeon crawler, though there is one hidden in it. Most of walking around LambdaMOO is exploring the many curiosities that users have provided there.
• Speaking of coding live inside the system, what's really interesting about LambdaMOO is that you're really manipulating a big database, and that many features are live coded from within the game itself. Everyone can experiment with adding rooms, though not everyone can program, though you may be able to get help from a certain buffer in the smoking room, if you can find it. (I just got access... I'll have to try things.)
• LambdaMOO is best known for a famous essay, but I think that essay is maybe not as interesting as it was in the 90s when online experiences were becoming a new thing? Still, the effect of the essay and the event it chronicles is still clear, including the way the world is introduced to you.
• Interesting to think of LambdaMOO in case of copyright and licensing, of which... well there seems to be no clear answer what is going on, but sharing seems not impeded by it.
• Being from the early 90s, it's amazing how much is here to explore, and how everything still lives. Since most of the world is pretty "low burn", that's nice for me, because I can just type in a command to look around somewhere every once in a rare while and leave LambdaMOO open in the background.
• I'm enjoying exploring the world, but I sense that LambdaMOO had a heyday, and while there are users connected, the sheer size of it vs the volume of connected users gives me the sense that today is not it. There's a certain amount of feeling of exploring a secret city, which is wonderful and sad. (It would be nice to have some friends who were interested in exploring with me, maybe I should cajole some into joining me.) It's an interesting piece of internet history, still here to explore, but since it's in a "living" database, what will happen when it shuts down? There won't be any Internet Archive to preserve the LambdaMOO memories, I fear. But maybe nothing is forever, anyway.

I have more, maybe crazier, thoughts on MUDs and maybe a way they could be more useful in this modern world. May I'll explore soon, but I think that the popularity of MUDs has diminished with World of Warcraft and Skyrim and the like, but maybe they have a place still if we can do some new things with them? And those are fairly hack-and-slash type systems, maybe MOO/MUD/MUSH systems have more to offer than popular MMO systems provide? Roguelikes seem more popular than ever, if anything... things can come back... maybe new interfaces are needed?...

In the meanwhile, if you drop by LambdaMOO, send me a message! I'm "paroneayea" there.

I found that I needed a topological sort algorithm in Guile, and it turns out that one already exists anyway in SLIB. Unfortunately, it seems that SLIB still doesn't work in Debian, and do I really want to depend on all that just for a topsort algorithm?

So here's SLIB's topsort algorithm, but with the hashmap switched over from slib's hashmap, which maybe makes it faster anyway. It was a pretty trivial change:

;;; "tsort.scm" Topological sort
;;; Copyright (C) 1995 Mikael Djurfeldt
;;;               2015 Chris Lemmer-Webber
;;; This code is in the public domain.

;;; The algorithm is inspired by Cormen, Leiserson and Rivest (1990)
;;; "Introduction to Algorithms", chapter 23

;;@code{(require 'topological-sort)} or @code{(require 'tsort)}
;;@ftindex topological-sort
;;@ftindex tsort

;;@noindent
;;The algorithm is inspired by Cormen, Leiserson and Rivest (1990)
;;@cite{Introduction to Algorithms}, chapter 23.

;;@body
;;@defunx topological-sort dag pred
;;where
;;@table @var
;;@item dag
;;is a list of sublists.  The car of each sublist is a vertex.  The cdr is
;;the adjacency list of that vertex, i.e. a list of all vertices to which
;;there exists an edge from the car vertex.
;;@item pred
;;is one of @code{eq?}, @code{eqv?}, @code{equal?}, @code{=},
;;@code{char=?}, @code{char-ci=?}, @code{string=?}, or @code{string-ci=?}.
;;@end table
;;
;;Sort the directed acyclic graph @1 so that for every edge from
;;vertex @var{u} to @var{v}, @var{u} will come before @var{v} in the
;;resulting list of vertices.
;;
;;Time complexity: O (|V| + |E|)
;;
;;Example (from Cormen):
;;@quotation
;;Prof. Bumstead topologically sorts his clothing when getting
;;dressed.  The first argument to @0 describes which
;;garments he needs to put on before others.  (For example,
;;Prof Bumstead needs to put on his shirt before he puts on his
;;tie or his belt.)  @0 gives the correct order of dressing:
;;@end quotation
;;
;;@example
;;(require 'tsort)
;;@ftindex tsort
;;(tsort '((shirt tie belt)
;;         (tie jacket)
;;         (belt jacket)
;;         (watch)
;;         (pants shoes belt)
;;         (undershorts pants shoes)
;;         (socks shoes))
;;       eq?)
;;@result{}
;;(socks undershorts pants shoes watch shirt belt tie jacket)
;;@end example

(define (tsort dag pred)
  (if (null? dag)
      '()
      (let* ((adj-table (make-hash-table)) ; SLIB was smarter about
                                           ; setting the length...
             (sorted '()))
        (letrec ((visit
                  (lambda (u adj-list)
                    ;; Color vertex u
                    (hashq-set! adj-table u 'colored)
                    ;; Visit uncolored vertices which u connects to
                    (for-each (lambda (v)
                                (let ((val (hashq-ref adj-table v)))
                                  (if (not (eq? val 'colored))
                                      (visit v (or val '())))))
                              adj-list)
                    ;; Since all vertices downstream u are visited
                    ;; by now, we can safely put u on the output list
                    (set! sorted (cons u sorted)))))
          ;; Hash adjacency lists
          (for-each (lambda (def)
                      (hashq-set! adj-table (car def) (cdr def)))
                    (cdr dag))
          ;; Visit vertices
          (visit (caar dag) (cdar dag))
          (for-each (lambda (def)
                      (let ((val (hashq-ref adj-table (car def))))
                        (if (not (eq? val 'colored))
                            (visit (car def) (cdr def)))))
                    (cdr dag)))
        sorted)))

(define topological-sort tsort)

Well there you go, it's in the public domain, so I guess you can just drop it straight into your project. Maybe Guile ought to have this included in its standard library? Lots of projects seem to reinvent it... maybe I should submit a patch...

I've been interested in Guix, the GNU functional package manager based on Guile and drawing much from Nix (though, I think, as a complete system, Guix is much more interesting... maybe a topic of its own post). After a great FOSDEM 2015, where amongst other things I got to speak to and hear from the super nice Guix developers (and even communicated much about the challenges with deploying libre webapps, a topic of one of the presentations I gave), I decided to finally install the thing.

Trouble is, if I'm mostly interested in playing with the package manager (there's now a complete Guix distro called the "Guix System Distribution", but it's a bit too alpha yet, and I'd like to play around in the system first anyhow), and I run Debian... but Guix isn't yet packaged for Debian (though I would like it to be!) But I hate... hate hate hate... doing a [make install]{.title-ref} of a package to my system. [make uninstall]{.title-ref} is so unreliable, and I like to keep my system as a combination of system packages (so, stuff through apt/dpkg) and stuff I've built but keep in my user's home directory. (Though, adding Guix to the mix now adds a third middle ground!) Plus, what if I want to hack on Guix, it seems like being able to have Guix use the code straight from my dev checkout is best, right?

Dave Thompson suggested that I do what he do: run Guix on top of Debian without a [make install]{.title-ref}, straight from the git repo. This was indeed what I wanted to do, but it took me a while to figure out the exact process to get that to work (especially in getting the emacs mode to work with this), so I'm documenting that here... maybe it can be helpful to someone else as well?

So we're mostly going to follow the install docs up until the point where we do a [make install]{.title-ref}, where we obviously won't. So:

# git clone the repo and cd into it
$ git clone git://git.savannah.gnu.org/guix.git
$ cd guix

# Install appropriate dependencies... insert your distro package manager here
# Not actually sure if -dev needed on libgcrypt20 or not :)
$ sudo apt-get install guile-2.0 libgcrypt20 libgcrypt20-dev build-essential \
    guile-gnutls guile-2.0-dev libsqlite3-dev pkg-config

# Build the package
$ ./bootstrap && ./configure && make

# Make the "worker users" and their group... this allows the daemon
# to offload package building while keeping things nicely contained
$ sudo groupadd guix-builder
$ for i in `seq 1 10`; do
    sudo useradd -g guix-builder -G guix-builder           \
                 -d /var/empty -s `sudo which nologin`          \
                 -c "Guix build user $i" --system          \
                 guix-builder$i;
  done

# Make the /gnu/store directory, where packages are kept/built
$ sudo mkdir -p /gnu/store
$ sudo chgrp guix-builder /gnu/store
$ sudo chmod 1775 /gnu/store

(Supposedly there's a way to run things without the extra users and [/gnu/store]{.title-ref}, but you lose a lot of benefits... personally it seems to me that having the store and the daemon is worth it.)

Okay, you're done! Now you should be able to run guix straight from your checkout. But you can't run it directly, you need to use the [./pre-inst-env]{.title-ref} script which has many environment variables set... after all, we aren't doing a make install, but guix needs to know where all our stuff is! First let's start the daemon... it runs as root, but don't worry, all package building happens as the guix-builder users we set up above:

$ sudo ./pre-inst-env guix-daemon --build-users-group=guix-builder

So, maybe leave that running in a screen session, or daemonize it, or something, but leave it running. For now we're just testing, so whatever.

This is optional, but you may want to "authorize" the Guix substitutes repository. (Thanks to [mthl]{.title-ref} in [#guix]{.title-ref} on freenode for pointing this out!) Otherwise, Guix will spend a lot of time rebuilding things itself that have already been built elsewhere:

$ sudo ./pre-inst-env guix archive --authorize < hydra.gnu.org.pub

Now we can actually run guix! (Maybe from another terminal?):

# a bunch of stuff should run by... but the package will install
$ ./pre-inst-env guix package -i hello

$ ~/.guix-profile/bin/hello
Hello, world!

Great! It ran. But okay... all three of the last commands were annoying. We probably want a way to launch these much more easily, so let's add aliases for [guix-daemon]{.title-ref} and [guix]{.title-ref}, as well as putting our user's guix profile on our [PATH]{.title-ref}. (Yes, that's right... different users can have different packages installed with guix pretty easily.) So, let's edit our [~/.bashrc]{.title-ref} and add the following lines:

function guix-enable() {
    # Guix stuff

    alias guix="~/devel/guix/pre-inst-env guix"
    alias guix-daemon="sudo ~/devel/guix/pre-inst-env guix-daemon --build-users-group=guix-builder"

    # add guix's bin to the path
    export PATH=$HOME/.guix-profile/bin:$PATH
    # and others
    export PYTHONPATH="$HOME/.guix-profile/lib/python3.4/site-packages"
    export GUILE_LOAD_PATH="$GUILE_LOAD_PATH:$HOME/.guix-profile/share/guile/site/2.0/"
    export GUILE_LOAD_COMPILED_PATH="$GUILE_LOAD_PATH:$HOME/.guix-profile/share/guile/site/2.0/"
}

You can [source ~/.bashrc]{.title-ref} if you have existing terminals open, or start new terminals / bash sessions... whatever. :) Anyway, it should now be much easier to run things:

$ guix-enable   # run this whenever you want to play with guix things
$ guix package -i xeyes
$ xeyes

Okay, great! So, now we have a reasonable hacking setup, right?

Well, assuming you want to be using Guix as the emacs of distros, you probably want to take advantage of Guix's great emacs integration.

Unfortunately, I found the docs on this subject frustratingly did not work with the method used above. Luckily, I got some help on IRC, and with the following additions to my ~/.emacs, things work:

(add-to-list 'load-path "/home/cwebber/devel/guix/emacs")

(setq guix-guile-program '("/home/cwebber/devel/guix/pre-inst-env" "guile"))

(setq guix-load-path "/home/cwebber/devel/guix/emacs")

(require 'guix-init)

Obviously, change directory names as appropriate.

Now [M-x guix-all-available-packages]{.title-ref} should work!

Hopefully I'll have more to say on why I think Guix is pretty interesting, but at the very least, it might be compelling to hear that Guix can be used as a sort of language-agnostic virtualenv. Pretty cool!

Hopefully that helps someone else out there!

Hey, so where did 2014 go, amirite guys? Amirite??

2014 in bulleted list form:

• Most stressful year of my life (and that includes the years I both worked full time and went to school full time), but not bad: high peaks and low valleys even out to a good year, but turbulent. Not just high and low events contributing to stress, also much stress has been ambient from ongoing and difficult events, but much of it not really befitting describing on this blog.

• MediaGoblin campaign went well, but I am tired of doing crowdfunding campaigns. Probably the last I will ever run... or at least the last for a long while. Nearly 5 months from start to wrapup of 60 hour high-stress weeks. But again, it went well! And hey, that video came out pretty great.

• Hiring Jessica was the smartest move we could have made, and I'm glad we made it.

• MediaGoblin federation work is going well; next release should make that clearer I hope.

• Both Jessica and I are on the W3C Social Working Group trying to standardize federation, and I'm excited about that.

• Hiring Jessica is great, but what to do about my own income? Happy to say I've started contracting for Open Tech Strategies who are great. Working under Karl Fogel is wonderful, and so is contracting for an org that mandates all code written there be free software. Plus, contracting 10 hours a week means I have plenty of other (including MediaGoblin) time left over.

• Also it's great to have a boss again who is reviewing my performance and seems happy with my work, especially when that boss is almost certainly more technically capable than I am; I forgot how much external affirmation from that is helpful in probably some base human way. I had some great bosses in the not too distant past, and while I think I'm a pretty decent boss to others, Morgan has pointed out that I am a really mean boss to myself.

• Despite ambient stress for both of us, Morgan and I's relationship goes well, maybe this year better than ever.

• Got nerdier, started playing tabletop role playing games with friends a lot. Board games too.

• Living in Madison is good.

• We are currently caring for a dog since another family member can't keep her where she is staying. Aside from temporary dog-sitting, I've never lived somewhere with a dog that I am caring for... it's interesting.

• The first half of the year was crazy due to the MediaGoblin campaign (again, I think it went great, and I had lots of help from great friends, just stressful), the second half crazy due to... well, a pile of things that are too personal for a blog (yeah I know I already said that). But everything came to a head right at the end of the year. This year burnt me the hell out.

• This made me pretty useless in December, and makes me feel terrible because I pushed vocally for a MediaGoblin release and failed to hold up my end of things to make it happen. I need to get back on track. This will happen, but in the meanwhile, I feel shitty.

• Burnout recovery has been productive; an odd thing to say maybe, but I seem to be getting a lot done, but not always on the things I think I should be.

• I feel more confident in myself as a programmer than before... I've always felt a large amount of impostor syndrome because I don't really have a computer science degree... I'm a community trained hacker (not to knock that, but it's hard to not feel insecure because of it).

  But this year I did some cool things, including getting patches in to a fun language, and I worked on an actor model system that I think has a hell of a lot of promise if I could just get the damned time for it. (If only I had time to solve error propagation and the inter-hive-communication demos...) I did every exercise in The Little Schemer (a real joy to work through) and I feel like hey, I finally understand how to write code recursively in a way that feels natural. And it turns out there is a MELPA-installable texinfo version of SICP and I've been slowly working my way through it when I'm too tired to do anything else but want to pretend to be productive (which has been a lot of the last month). Still so much to learn though, but I appreciate the bottomless well aspect of programming.

• Aside from the MediaGoblin campaign video, not a lot of artwork done this year. Hrm.

• A couple of friends this year have made the "I've been doing nothing but python webdev for years and I need to mix it up" and to those friends: I hear you. Maybe hence the above?

• Aside from MediaGoblin I've been doing a lot more chipping away at tiny bits of some free software projects, but maybe nothing significant enough to blog about yet, but there's a deployment system in there and a game thing and some other stuff. Nothing MediaGoblin sized, though. (Whew!)

• Enjoying learning functional reactive programming (and expanding my understanding of Scheme) with Sly. Unfortunately still under-documented, but it's getting better, and davexunit is answering lots of questions for me. I might write a tutorial, or tutorials, soon.

• Another ascii art logo for a FOSS project I made got vectorized and made way better than my original version, but that deserves its own post.

• I continue to be able to work on free software full time, which is great.

I feel like the start of 2015 has already been moving upward, and has been much less stressful than the end of 2014. I hope that curve can keep moving up. And I hope I can keep up what I feel, despite me nearly going insane from various facets of it, is a fairly productive life.

Thought of the morning: people often say "we have variables and comments because programming languages are for humans, not computers". But if computers were really at the point where they were able to program themselves, and I mean really do it (even invent and code new things, and I don't mean genetic algorithm bullshit, I mean thinking about design... so this also means code as more than just "learning", but actually planning and programming something new), would they need variable names and comments?

My thinking is: yes, or they'd need something like it. If you don't have this, this means you're effectively reverse engineering "purpose" in the codebase all the time, which can be both expensive and faulty. I think any AI that's not some ~dumb application of known heuristics will need to be able to "think" about the code at point, and knowing the reason for a code change is important. So of course relevant information should be recorded in that portion of the code.

Now, does that mean something as messy as English will be used (as the majority of present code is written in English)? I doubt it. Probably something like Lojban will be used. Maybe it will not even be plaintext code: it could be machine-readable, machine-contemplatable code with "byte compiled lojban", or similar.

Relatedly, in the (glorious???) future where machines can think and design programs, assuming enough resources exist to keep said machines running, humans will have to interface with computers on a computer's level more often. Will Lojban as a second language be mandated in schools?

(On that note, maybe I should really learn Lojban... :))

I'm happy to see the FSF has revitalized the high priority free software projects list.

The FSF is calling for input, so consider the following my "high priority wishlist": these are all things I wish I had nicer representations in free software, and the lack thereof is hindering user freedom in some urgent way.

Server deployability tools

Deployment needs to become easier if we expect people to have any amount of digital autonomy. Processes need to become generalized, abstracted. I have written some ideas on this. I am happy to talk to anyone interested in improving this space... :)

Federation tools

Obviously somewhat self-serving, though my very reason for working on federation tools is due to my belief that a lack of federation tooling leads to many problems. Those arguments already well understood by anyone reading this site, so I won't go into details.

Implementation of the Social WG standards would be a good direction to take. Making this implementation reusable would be helpful.

"Unity competitor" level game engine

I've also written before on why I believe that games matter to free software and free culture.

But I'll particularly emphasize here: 1) proprietary games are bringing DRM to otherwise free software operating systems via steam, 2) games are currently the world's most popular medium, so if expression here is particularly hard, this is not good and 3) games are a great way to recruit new hackers, as this is the dream of many.

Assuming you buy those arguments, we are missing out on many opportunities of expression by the sheer amount of work required to get the basics of most games running. In the proprietary world, there is a game engine called Unity that seems popular not just for developers but also for artists. What is free software's answer?

Though it will surely come up, I'm not confident in the Blender Game Engine. Godot looks promising.

(As a side note, both of the following systems are fairly low-level at the moment and thus do not fulfill the above "full featured and easy to use", but some interesting things are happening in the lands of both Sly and m.grl (both by friends of mine)... I hope more happens with these.)

Password management tooling

Managing passwords is hard but important, and the current state of affairs in free software is a sorry one. Many users are turning to "LastPass", which astounds me in a post-Snowden, post-Lavabit world. How to provide a way to store passwords safely and easy to use way?

KeePassX and others which require manually copying around passwords I think are simply not usable enough given the massive volume of times users need to enter passwords.

I currently use assword, as "assword gui", when bound to a key in your window manager, gives a simple-ish prompt, with passwords dumped into whatever textbox previously had focus in X11. It also pleasantly stores passwords in a simple format: gpg encrypted json. Not bad, but there's no nice "management" interface, and there's no "mobile" (read: Android/Replicant) client.

Easier encryption

It's great that we have GnuPG, but usage is hard. How to make easer? Maybe something like monkeysign could help web of trust things, but not only web of trust things are hard. SSL is also a great way to lose hair.

Relatedly, GPG could use your help, funding-wise! Consider making a donation.

Free phones and mobile devices

Is this software? Sort of, it also ties in with hardware.

Replicant is the best thing we have for phones. (I would love to see a resurrection of Maemo 5, but I realize I am in the minority here.)

Wearable computing

Where is free software's competitor to Google Glass? Even if it isn't cool right now, some competitor will make pervasive computing popular. Do we really want to wait for the iEye before we start worrying that we don't have a solution? This, like free phones, requires both hardware and software.

Circuit printing pipeline

Libre hardware which can print circuits could possibly get us well on our way towards building hardware that's safe, reproducible, verifiable. It won't help with chips, still will require lots of soldering, but maybe it could help.

Easy, peer to peer filesharing between friends

The return of peer to peer? Yes indeed. I see this one is already on the list though. :)

I love nonprofits and activist orgs, and I think political campaigns should send emails to their base asking for support.

That said, for the last few years, I've gotten angry at nearly every email subject I've gotten from activist organizations. You've probably gotten the same, but I've collected some samples:

• re: The site crashed!
• Um...
• re: Un-freaking-believable
• Argh!
• Please please please
• Forward this!
• Forward this →
• This is dangerous:
• Holy moly!!!
• re: Corrupt?
• re: Comcast (like a thousand of these)
• Pancakes and syrup (yes, a real subject sent for non-breakfast-invitation reasons)

And the list goes on, and on. My "Political Campaigns" folder is a joke.

Apparently this started in Obama's re-election campaign and was found to be successful in terms of getting both eyeballs and dollars.

Is it still successful? If it was a novel trick, it isn't anymore. What's worse, these subject lines are awfully similar to two groups of emails I hate to get: the classic and loathed "ha ha forward this to friends" email from relatives, and spam. Which means many of these I automatically mark as read or accidentally misfile as spam.

But I'm tired of getting these emails. If you're an activist org, and you send an email to me like this, I'm unsubscribing. And as immature as it might be to say, I'm unlikely to donate to your campaign again, assuming I ever did.

Political/activist orgs: don't treat me like an idiot. Prove my brain energy is worth your time.

Hello everyone! It's that time of the year again, and many people are considering: who they are going to donate to this year?

If you are a supporter of computing freedom, there are a lot of great places to put your money. The FSF and EFF are great places, and probably the most commonly recommended (also worth your money!), though this year I'm going to suggest you also consider the Software Freedom Conservancy. They just launched a membership program today, and I recommend you join. I just did!

Whether you realize it or not, you probably depend day to day on some software that Conservancy helps support... which it turns out is a lot: Git, Inkscape, Mercurial, Wine, BusyBox... these are just a few of the many projects that are members of Conservancy. And Conservancy does a lot of stuff for their member projects! (Not to mention a number of things that benefit free software that don't even hit that list, including Conservancy employees spreading information on best practices and participating in efforts to protect the legal freedoms of all free software... I recommend subscribing to Bradley Kuhn and Karen Sandler's podcast Free as in Freedom... I've learned a lot from there, and am learning more all the time!) What I'm trying to say here is: donating to Conservancy is a real bang for your buck. :)

So this holiday season, join me in helping make Software Freedom Conservancy's membership program a success! Your user freedoms will thank you for it!

Hello everyone! Time for a bit of a diversion towards one of my favorite projects, Hy! (I'm an occasional committer, but the main mastermind behind the project is my good friend Paul Tagliamonte.) For those of you who don't know, Hy is a Lisp that transforms into the Python AST. Even more fun: you can import .hy files in .py files and .py files in .hy files! Crazy!

Now, when many people hear that, they say, "Huh what, why on earth would you do such a thing?" The usual response is something like, "Because it's fun!" But today, dear readers, I am going to show you a real... dare I say practical reason for using Hy. Because a cool feature just landed in Hy: a backport of "yield from" to Python 2.

Let's back up a bit. First, you might not know what "yield from" is or why it's cool. Well, Python has this thing called coroutines which allow you to do cool things, including suspending and resuming functions, which it turns out is really great for writing asynchronous code (sure, just wake me back up when we get the next network stanza, eh buddy?). Once you provide the ability to nest together coroutines by "delegating to subgenerators" (what "yield from" does that "yield" does not), this stuff starts to get really powerful. This feature is so useful that it's the basis of maybe the world's coolest asynchronous programming environment, asyncio. Only one problem: "yield from" didn't exist until Python 3.3... which means you can't use it with Python 2. Bummer!

Or can you? Time for our second bit of context. Ever hear of a Lisp programmer talk about something called a "macro"? No? Okay, think harder. Maybe it was in one of those conversations where you were talking about your favorite new your-pet-language feature, and the Lisp hacker was like, "Oh that's cute... yeah Lisp had that decades ago." And then you got really mad and brought up a bunch more features, and the Lisp hacker kept saying that Lisp had them before you were born, and "Oh yeah, and whatever features Lisp doesn't have, you can add really fast because Lisp has macros. You can basically program any feature with macros." Maybe you asked them, what the heck is a macro, and they said something inane like "it's a feature where you can program other features, or write code that writes code", but you barely remember, because at that point you just wanted to punch them in their smug little face. (And besides, you wondered, if you have higher order functions, isn't that enough?)

Well friends, today it is my face that you will want to do the punching to, because I'm about to show you how cool macros are and why having them makes Hy so awesome. But, after the face punching thing, you'll also thank me. (Also, please don't punch me in the face, this blogpost is not consent for face-punching.)

Enough with the talk. Time for examples! Let's look at some code. Say you have this Python 3.3+ code:

from awesomelib import IrcBot, bake_cookie, async

def irc_to_cookies(**connection_stuff):
    our_bot = IrcBot(**connection_stuff)
    yield from our_bot.open_connection()

    while True:
        message = yield from our_bot.get_next_message()

        if message.command == "bake_cookie":
            yield from async(bake_cookie())

Groovy. In our example above, we built a cookie baker that can be plugged into our awesomelib asynchronous network library and cookie baking pipeline system. (And of course, we wrote it in Hy, because we love Hy, and you can still run Hy code in vanilla Python.) We're feeling pretty good about this. We kind of wish we could run it in Python 2.X still, but Python 3.3+ is the future anyway, and no use worrying about the past really... right?

Our Hy example looks pretty similar:

(defn irc-to-cookies [&kwargs connection-stuff]
  (setv our-bot (apply IrcBot [] connection-stuff))
  (yield-from (.open-connection our-bot))

  (while True
    (setv message (yield-from (.get-next-message our-bot)))

    ;; If an irc user gives the "bake_cookie" command,
    ;; put a cookie in our ez_bake oven
    (if (= message.command "bake_cookie")
      (yield-from (async (bake-cookie))))))

But there's something magical... this code makes use of yield-from, which in Python 3.3+ Hy just uses the actual real built in "yield from" (or more accurately, ast.YieldFrom). But what about Python 2? Mere higher ordered function magic can't save us here. We need a way to implement a new feature.

Except oh right, this is a lisp, and we have macros! So why not write a macro for yield-from?

And it turns out that's exactly what paultag did:

(if-python2
  (defmacro/g! yield-from [expr]
    `(do (import types)
         (setv ~g!iter (iter ~expr))
         (setv ~g!return nil)
         (setv ~g!message nil)
         (while true
           (try (if (isinstance ~g!iter types.GeneratorType)
                  (setv ~g!message (yield (.send ~g!iter ~g!message)))
                  (setv ~g!message (yield (next ~g!iter))))
           (catch [~g!e StopIteration]
             (do (setv ~g!return (if (hasattr ~g!e "value")
                                     (. ~g!e value)
                                     nil))
               (break)))))
           ~g!return))
  nil)

This simple macro above is an implementation of yield-from which works in Python 2. The macro is more or less a function that writes new code to be expanded in place... allowing us to use basic building blocks of the language to build more complex features. Since in Lisp, code is a very simple, manipulatable data structure (lists!), we can literally write out code that writes code without too much trouble. (There's some magic going on with the ` character above, called backquoting... but it's best to read a tutorial on macros if it's not clear to you how the backquote is building the list of code there.) Hey look... we just brought a feature back to the future... as long as we're writing code in Hy, we can do subgenerator delegation with yield-from. Cool! That sure makes coroutines a lot more useful to those of us living in the past.

So wait, does this mean you can now use asyncio with Python 2? Well, not quite... asyncio is written in normal Python syntax, which means that it's using "yield from", not our more versatile "yield-from", and the library itself isn't written to support Python 2.7. So, no. (But, if asyncio was written in Hy, we could, even though Python 2.7 doesn't have "yield from"!)

The real goal of this article isn't to convince you to start backporting features to Python 2.X via Hy, though. Really, Python 3 is the future, write Python 3 code! But the point here is to get you thinking about how having macros allows you to implement new features now! Why wait for the features of Python 4.X? In Hy you can have them now! (Or even start prototyping them today!)

And that's worth getting excited about. And once you realize that, it's a bit easier to understand where lispers are coming from when they nerd out about how cool macros are. (Even if you still want to punch us in the face.)

(PS: think this is pretty cool? Hy is a really welcoming community, and there's a lot of fun stuff to do! Learn about language implementation and learn about lisp in a paradoxically fun and pythonic environment! We'd love to have you join in hacking with us!)