Federation and GNU

Christopher Allan Webber

cwebber@dustycloud.org

2015-03-19 Sat

Introduction

Who am I?

  • Lead MediaGoblin developer!
  • Free software and free culture activist
    • MediaGoblin co-founder / lead dev
    • Now: full time MediaGoblin
    • Formerly: CC, PCF, other things
  • Working on W3C Social WG for federation standard

What's MediaGoblin?

MediaGoblin is a free software media publishing system anyone can use and run!

But…

Decentralization comes with a challenge

Say you uploaded an awesome photo of a mountain on your MediaGoblin instance…

How to share with friends on other servers?

Federate it!

What is federation?

Why do you care?

The present reality of network freedom isn't pretty…

  • Corporate control of the web
  • NSA spying
  • Freedom for developers, but not for users

So, let's build something better :)

What does federation mean for MediaGoblin?

Mostly federating social activities:

  • Subscribing to someone else's media uploads
  • Commenting
  • Favoriting
  • Note: doesn't necessarily mean "mirroring" media! (though maybe a "diskgobbling" plugin could be added)

How might you use it?

Propagating most "activities" across servers:

  • Sending messages between users privately
  • Posting messages publicly
  • Scheduling calendar events
  • Posting media

A history of federation

Pre-web federation

Federation isn't a new concept.

Some older federation standards:

  • Email
  • XMPP/Jabber
  • OpenID (So, not just sharing data)

The web in the present

Great image from Michał 'Rysiek' Woźniak on active, community-engaged federation projects. Except, add Pump.io!

The Original Series

As long as we're making Star Trek metaphors…

Original cast

  • Main cast:
    • GNU Social, formerly StatusNet: OStatus
    • Diaspora: (OStatus, mostly)
    • Friendica: Just about everything (incl OStatus)
  • Recurring characters:
    • BuddyCloud (using XMPP on backend)
    • RStatus (Ruby OStatus microblog implementation)
    • Elgg
    • Others I'm sure I've forgotten to mention and will be chastized for

OStatus

  • Several authors, though most prominently Evan Prodromou
  • The most "successful" of the Original Protocols
  • Kind of a meta-protocol, weaving together:
    • Webfinger for identity
    • Atom + Activitystreams 1.0 XML representation
    • PubSubHubbub to push out public updates
    • Salmon for private notifications / messaging
  • A good start, but…

OStatus (cotd)

  • A big meta-spec means much more to learn
  • Salmon is hard to understand / implement right, private messaging never really worked
  • Non-protocol "issues": flagship StatusNet site "identi.ca" grew hard to scale, hard to fund with sponsor company gone
  • Evan Prodromou started working on Pump.IO and the Pump API during shift
  • OStatus status:
    • Still used by some key players
    • But protocol design itself stagnant

The Next Generation

The cast

  • Interesting stuff at:
    • Pump.IO: the Pump API (by StatusNet's founder)
    • Red Matrix: Zot (by Friendica's founder)
    • Mayyyyyybe Tent.io
  • Coming up in next season:
    • The W3C Social WG

Some similarities

  • Exchanging json activities to various endpoints seems common
  • Private delivery worked in from the start

Focus: Pump.io and the Pump API

MediaGoblin decided to follow Pump for its federation work. But why?

  • Easy to understand: Mostly just pushing json snippets from a few simple endpoints
  • Led by Evan Prodromou, who has done much leadership here
  • Local expertise: PyPump contributor part of the MediaGoblin project

More pump.io tech

  • Posting JSON encoded Activitystreams 1.0 objects
  • Each "user" has inbox/outbox endpoints
  • Webfinger for email-like identity lookup
  • OAuth for authentication between servers

That's "mostly it", see more at the https://github.com/e14n/pump.io/blob/master/API.md

What does a Pump post look like?

{
    "id": "http://coding.example/api/activity/bwkposthw",
    "actor": {
        "id": "acct:bwk@coding.example",
        "displayName": "Brian Kernighan",
        "objectType": "person",
        "url": "http://coding.example/bwk"
    },
    "verb": "post",
    "object": {
        "id": "http://coding.example/api/note/helloworld",
        "content": "Hello, World!"
        "objectType": "note"
    },
    "published": "1973-01-01T00:00:00"
}

Subject, Predicate, Object!

It's pretty simple, easy to unerstand, and solid tech.

But is it ready? (p. 1)

But is it ready? (p. 2)

We're in "season one": good foundations, but a hard sell to newcomers.

A fractured federation

The protocol is the glue of federation here, and yet we don't seem to agree on what protocol to use…

We now have a smattering of incompatible protocols:

  • OStatus
  • Zot
  • XMPP
  • Pump
  • Tent

Even GNU does not "agree"

Is a better future possible?

W3C Social Working Group!

Let's make this official, yo

Status

  • Standardizing on Activitystreams 2.0 (currently in working draft, but solidifying)
  • Thorough analysis of requirements from other social networks
  • About to start standardizing "Social API" (client to server communication)
  • Then on to formalize federation (server to server communication)
  • Some philosophical differences to resolve though ("make it easy" vs "solid engineering")

Lessons learned from MediaGoblin federating

What can GNU do?

Pioneer in federated web applications

Beyond federation?


The demonization and confinement of peer to peer

File sharing isn't evil, but peer to peer isn't just for file sharing either…

New ideas in peer to peer

Federation over peer to peer?

Everything that's old is GNU again

Older systems may provide good ideas for newer tech! (The dreams of the 90s are alive in federation???)

Actually a huge topic of research in the 1990s

But back to MediaGoblin

  • 0.8.0 is right around the corner, supports client->server Pump API
  • server to server (federation) stuff is in progress, expected by next major release after

It's happening!

Thanks to Jessica

Thanks to you

Thanks again (for listening!)

Credits

  • Space colony art by NASA Ames Research Center, US Gov public domain http://settlement.arc.nasa.gov/70sArt/art.html
  • Pirate Bay logo is "Kopimi", taken from Wikipedia
  • MediaGoblin artwork where present mostly by me, CC BY-SA 4.0!
  • Bitcoin logo CC0 1.0 from bitcoin reference client
  • Telehash logo in the public domain, see telehash.org git repo

Credits (fair use!)

Lots of pictures from Star Trek, which is proprietary and by Universal Pictures, but fair use and stuff.

Sources: 

- http://asena42.blogspot.com/2011/08/live-long-and-prosper.html
- http://tng.trekcore.com/gallery/albums/s1/1x17/homesoil004.jpg
- https://i.imgur.com/5VCDtNw.jpg
- https://startrektherewatch.files.wordpress.com/2014/11/skinofevil_hd_386.jpg?w=1180&h=885

Plus this thing by Rysiek: 

- http://events.ccc.de/congress/2014/wiki/Assembly:The_Federation

Contact info / Questions?

This talk CC BY-SA 4.0 International