Federation and Python

Christopher Allan Webber

cwebber@dustycloud.org

2015-01-31 Sat

Introduction

Who am I?

  • Lead MediaGoblin developer!
  • Free software and free culture activist
    • MediaGoblin co-founder / lead dev
    • Now: full time MediaGoblin
    • Formerly: CC, PCF, other things
  • Working on W3C Social WG for federation standard

What's MediaGoblin?

MediaGoblin is a free software media publishing system anyone can use and run!

But…

Decentralization comes with a challenge

Say you uploaded an awesome photo of a mountain on your MediaGoblin instance…

How to share with friends on other servers?

Federate it!

What is federation?

Why do you care?

The present reality of network freedom isn't pretty…

  • Corporate control of the web
  • NSA spying
  • Freedom for developers, but not for users

So, let's build something better :)

What does federation mean for MediaGoblin?

Mostly federating social activities:

  • Subscribing to someone else's media uploads
  • Commenting
  • Favoriting
  • Note: doesn't necessarily mean "mirroring" media! (though maybe a "diskgobbling" plugin could be added)

How might you use it?

Propagating most "activities" across servers:

  • Sending messages between users privately
  • Posting messages publicly
  • Scheduling calendar events
  • Posting media

A history of federation

Pre-web federation

Federation isn't a new concept.

Some older federation standards:

  • Email
  • XMPP/Jabber
  • OpenID (So, not just sharing data)

The web in the present

Great image from Michał 'Rysiek' Woźniak on active, community-engaged federation projects. Except, add Pump.io!

The Original Series

As long as we're making Star Trek metaphors…

Original cast

  • Main cast:
    • StatusNet (now GNU Social): OStatus
    • Diaspora: (OStatus, eventually, sorta?)
    • Friendica: Just about everything (incl OStatus)
  • Recurring characters:
    • BuddyCloud (using XMPP on backend)
    • RStatus (Ruby OStatus microblog implementation)
    • Elgg
    • Others I'm sure I've forgotten to mention and will be chastized for

OStatus

  • Several authors, though most prominently Evan Prodromou
  • The most "successful" of the Original Protocols
  • Kind of a meta-protocol, weaving together:
    • Webfinger for identity
    • Atom + Activitystreams 1.0 XML representation
    • PubSubHubbub to push out public updates
    • Salmon for private notifications / messaging
  • A good start, but…

OStatus (cotd)

  • A big meta-spec means much more to learn
  • Salmon is hard to understand / implement right, private messaging never really worked
  • Non-protocol "issues": flagship StatusNet site "identi.ca" grew hard to scale, hard to fund with sponsor company gone
  • Evan Prodromou started working on Pump.IO and the Pump API during shift
  • OStatus status:
    • Still used by some key players
    • But protocol design itself stagnant

The Next Generation

The cast

  • Interesting stuff at:
    • Pump.IO: the Pump API (by StatusNet's founder)
    • Red Matrix: Zot (by Friendica's founder)
    • Mayyyyyybe Tent.io
  • Coming up in next season:
    • The W3C Social WG

Some similarities

  • Exchanging json activities to various endpoints seems common
  • Private delivery worked in from the start

Focus: Pump.io and the Pump API

MediaGoblin decided to follow Pump for its federation work. But why?

  • Easy to understand: Mostly just pushing json snippets from a few simple endpoints
  • Led by Evan Prodromou, who has done much leadership here
  • Local expertise: PyPump contributor part of the MediaGoblin project

More pump.io tech

  • Posting JSON encoded Activitystreams 1.0 objects
  • Each "user" has inbox/outbox endpoints
  • Webfinger for email-like identity lookup
  • OAuth for authentication between servers

That's "mostly it", see more at the Pump API spec document

What does a Pump post look like?

{
    "id": "http://coding.example/api/activity/bwkposthw",
    "actor": {
        "id": "acct:bwk@coding.example",
        "displayName": "Brian Kernighan",
        "objectType": "person",
        "url": "http://coding.example/bwk"
    },
    "verb": "post",
    "object": {
        "id": "http://coding.example/api/note/helloworld",
        "content": "Hello, World!"
        "objectType": "note"
    },
    "published": "1973-01-01T00:00:00"
}

Subject, Predicate, Object!

It's pretty simple, easy to unerstand, and solid tech.

But is it ready? (p. 1)

But is it ready? (p. 2)

We're in "season one": good foundations, but a hard sell to newcomers.

A fractured federation

The protocol is the glue of federation here, and yet we don't seem to agree on what protocol to use…

We now have a smattering of incompatible protocols:

  • OStatus
  • Zot
  • XMPP
  • Pump
  • Tent

Is a better future possible?

W3C Social Working Group!

Let's make this shit official

Status

  • Standardizing on Activitystreams 2.0 (currently in working draft, but solidifying)
  • Thorough analysis of requirements from other social networks
  • About to start standardizing "Social API" (client to server communication)
  • Then on to formalize federation (server to server communication)
  • Probably will be very Pump-like… things seem to be moving in that direction.

What's a pythonista to do?

PyPump

  • Object oriented
  • Pythonic
  • Clear encapsulation of activities, etc
  • Simplifies auth

How about a real demo!

How the Python world can help

  • Contribute / use PyPump
  • Get involved in implementing federation into your web applications
    • The Pump API
    • What comes out of the Social WG
  • Help us build fork of PyPump that migrates towards Social WG standardization
  • Interested in helping build a reference implementation of the Social WG spec? We should talk!

Thanks

Credits

Credits (fair use!)

Lots of pictures from Star Trek, which is proprietary and by Universal Pictures, but fair use and stuff.

Sources:

Plus this thing by Rysiek:

Contact info / Questions?

This talk CC BY-SA 4.0 International