# -*- mode: org; org-html-inline-images: nil; -*-

#+TITLE:     Federation and GNU
#+AUTHOR:    Christopher Allan Webber
#+EMAIL:     cwebber@dustycloud.org
#+DATE:      2015-03-19 Sat

#+OPTIONS: reveal_center:t reveal_progress:t reveal_history:nil reveal_control:t
#+OPTIONS: reveal_mathjax:t reveal_rolling_links:t reveal_keyboard:t reveal_overview:t num:nil
#+OPTIONS: reveal_width:1200 reveal_height:800
#+OPTIONS: toc:nil
#+REVEAL_MARGIN: 0.06
#+REVEAL_MIN_SCALE: 1
#+REVEAL_MAX_SCALE: 2.5
#+REVEAL_TRANS: cube
#+REVEAL_THEME: night
#+REVEAL_HLEVEL: 2
#+REVEAL_HEAD_PREAMBLE: <meta name="description" content="Federation and Python">


* Introduction
** Who am I?

#+REVEAL_HTML: <img style="float: right;" src="../../network_freedom/static/libreplanet_toon_chris.png" />

# #+ATTR_REVEAL: :frag roll-in
 - Lead MediaGoblin developer!
 - Free software and free culture activist
   - MediaGoblin co-founder / lead dev
   - Now: full time MediaGoblin
   - Formerly: CC, PCF, other things
 - Working on W3C Social WG for federation standard

** What's MediaGoblin?

#+REVEAL_HTML: <img style="height: 550px;" src="../../mediagoblin/static/mediagoblin_mascot.png" />

MediaGoblin is a free software media publishing system anyone can use
and run! (And a GNU project!)

But...

** Decentralization comes with a challenge

#+REVEAL_HTML: <img style="height: 550px;" src="../static/federation-user_adds_image.png" />

Say you uploaded an awesome photo of a mountain on your MediaGoblin
instance...

*** How to share with friends on other servers?

#+REVEAL_HTML: <img style="height: 550px;" src="../static/federation-unsynced.png" />

*** Federate it!

#+REVEAL_HTML: <img style="height: 550px;" src="../static/federation-synced.png" />

** What is federation?

#+REVEAL_HTML: <img style="height: 600px;" src="../../network_freedom/static/purple_federation.png" />

#+BEGIN_NOTES
 - loose term, means interconnecting separate servers.
 - you want to tear down that centralization wall?
 - need a way to bring the same experiences they can have on a
   centralized server.
#+END_NOTES

** Why do you care?

#+REVEAL_HTML: <img style="border: none; height: 390px;" src="../../network_freedom/static/nsa-eagle-mildalterations.png" />

The present reality of network freedom isn't pretty...

 - Corporate control of the web
 - NSA spying
 - Freedom for developers, but not for users

So, let's build something better :)

** What does federation mean for MediaGoblin?

# Various activities need to be sync'ed across servers

Mostly federating social activities:

 - Subscribing to someone else's media uploads
 - Commenting
 - Favoriting
 - Note: doesn't necessarily mean "mirroring" media!
   (though maybe a "diskgobbling" plugin could be added)

** How might you use it?

Propagating most "activities" across servers:

 - Sending messages between users privately
 - Posting messages publicly
 - Scheduling calendar events
 - Posting media

* A history of federation

** Pre-web federation

Federation isn't a new concept.

Some older federation standards:

#+ATTR_REVEAL: :frag roll-in
 - Email
 - XMPP/Jabber
 - OpenID  (So, not just sharing data)

** The web in the present

#+REVEAL_HTML: <img style="border: none; height: 570px;" src="../static/nonfree/Founding_Species_of_the_Federation.png" />

Great image from Michał 'Rysiek' Woźniak on active, community-engaged
federation projects.  Except, add Pump.io!

** The Original Series

#+REVEAL_HTML: <img style="border: none; height: 570px;" src="../static/nonfree/star_trek_the_original_series_season_2_10.jpg" />

As long as we're making Star Trek metaphors...

*** Original cast

# #+ATTR_REVEAL: :frag roll-in
 - Main cast:
   - GNU Social, formerly StatusNet: OStatus
   - Diaspora: (OStatus, mostly)
   - Friendica: Just about everything (incl OStatus)
 - Recurring characters:
   - BuddyCloud (using XMPP on backend)
   - RStatus (Ruby OStatus microblog implementation)
   - Elgg
   - Others I'm sure I've forgotten to mention and will be
     chastized for

*** OStatus

# TODO: OStatus logo here

#+ATTR_REVEAL: :frag roll-in
 - Several authors, though most prominently Evan Prodromou
 - The most "successful" of the Original Protocols
 - Kind of a meta-protocol, weaving together:
   - Webfinger for identity
   - Atom + Activitystreams 1.0 XML representation
   - PubSubHubbub to push out public updates
   - Salmon for private notifications / messaging
 - A good start, but...

*** OStatus (cotd)

#+ATTR_REVEAL: :frag roll-in
 - A big meta-spec means much more to learn
 - Salmon is hard to understand / implement right, private messaging
   never really worked
 - Non-protocol "issues": flagship StatusNet site "identi.ca" grew
   hard to scale, hard to fund with sponsor company gone
 - Evan Prodromou started working on Pump.IO and the Pump API during
   shift
 - OStatus status:
   - Still used by some key players
   - But protocol design itself stagnant

#+BEGIN_NOTES
OStatus protocol work is pretty much stagnant at this point, though
several communities actively using the protocol.

MediaGoblin was going to originally use OStatus, but... now we come
to:
#+END_NOTES

** The Next Generation

#+REVEAL_HTML: <img style="height: 550px;" src="../static/nonfree/tng_homesoil004.jpg" />

*** The cast

 - Interesting stuff at:
   - Pump.IO: the Pump API  (by StatusNet's founder)
   - Red Matrix: Zot  (by Friendica's founder)
   - Mayyyyyybe Tent.io
 - Coming up in next season:
   - The W3C Social WG

*** Some similarities

 - Exchanging json activities to various endpoints seems common
 - Private delivery worked in from the start

*** Focus: Pump.io and the Pump API

MediaGoblin decided to follow Pump for its federation work.  But why?

 - Easy to understand: 
   Mostly just pushing json snippets from a few simple endpoints
 - Led by Evan Prodromou, who has done much leadership here
 - Local expertise: PyPump contributor part of the MediaGoblin project

*** More pump.io tech

 - Posting JSON encoded Activitystreams 1.0 objects
 - Each "user" has inbox/outbox endpoints
 - Webfinger for email-like identity lookup
 - OAuth for authentication between servers

That's "mostly it", see more at the
[[https://github.com/e14n/pump.io/blob/master/API.md][Pump API spec document]]

*** What does a Pump post look like?

#+BEGIN_SRC python
{
    "id": "http://coding.example/api/activity/bwkposthw",
    "actor": {
        "id": "acct:bwk@coding.example",
        "displayName": "Brian Kernighan",
        "objectType": "person",
        "url": "http://coding.example/bwk"
    },
    "verb": "post",
    "object": {
        "id": "http://coding.example/api/note/helloworld",
        "content": "Hello, World!"
        "objectType": "note"
    },
    "published": "1973-01-01T00:00:00"
}
#+END_SRC

Subject, Predicate, Object!

It's pretty simple, easy to unerstand, and solid tech.

*** But is it ready? (p. 1)

#+REVEAL_HTML: <img style="height: 600px;" src="../static/pump_web_and_pumpa.png" />

#+BEGIN_NOTES
Web api sucks, "but the web api is just one possible client"

Good point, but not an easy introduction to family and friends
#+END_NOTES

*** But is it ready? (p. 2)

#+REVEAL_HTML: <img style="height: 550px;" src="../static/nonfree/skinofevil.jpg" />

We're in "season one": good foundations, but a hard sell to newcomers.

#+BEGIN_NOTES
Basically, I think we're on season 1 of Next Generation.  We've got
Picard, but if this is all we have, it's going to be hard to get
people into the series.

Luckily, we've got good foundations.
#+END_NOTES

** A fractured federation

The protocol /is/ the glue of federation here, and yet we don't seem
to agree on what protocol to use...

We now have a smattering of incompatible protocols:

 - OStatus
 - Zot
 - XMPP
 - Pump
 - Tent

Even GNU does not "agree"

** Is a better future possible?

# Public domain "future" inventions images

# http://publicdomainreview.org/collections/a-journey-in-other-worlds-a-romance-of-the-future-1894/
# http://publicdomainreview.org/collections/leaving-the-opera-in-the-year-2000/

# http://publicdomainreview.org/collections/space-colony-art-from-the-1970s/ <- us gov public domain

#+REVEAL_HTML: <img style="height: 600px;" src="../static/gerard_oneill_spacecolony.jpeg" />


*** W3C Social Working Group!

#+REVEAL_HTML: <img style="height: 600px;" src="../static/socialwg_wiki_screenshot.jpg" />

Let's make this official, yo

*** Status

# TODO clarify this

 - Standardizing on Activitystreams 2.0 (currently in working draft, but
   solidifying)
 - Thorough analysis of requirements from other social networks
 - About to start standardizing "Social API" (client to server
   communication)
 - Then on to formalize federation (server to server communication)
 - Some philosophical differences to resolve though
   ("make it easy" vs "solid engineering")

** Lessons learned from MediaGoblin federating

#+REVEAL_HTML: <img style="height: 600px;" src="../static/goblinverse.png" />

#+BEGIN_NOTES
 - Estimated time to port Pump API to MediaGoblin: 1 full year of
   expert, dedicated engineering
 - Why so long?
   - Database migrations are *really* tough to adjust to add
     decentralization to a previously centralized codebase...
   - Most UI and etc systems not set up for it
   - Just generally a lot of work to plan in
 - Planning federation from day one would have been simpler
   (hindsight is 20/20 though)
#+END_NOTES

* What can GNU do?

#+BEGIN_NOTES
GNU has always been a leader in pioneer in freeing users where constrained
#+END_NOTES


** Pioneer in federated web applications

#+REVEAL_HTML: <img style="height: 600px;" src="../static/gavroche_gnu-sticker.png" />

#+BEGIN_NOTES
 - There really aren't enough people working on federated webapps
 - Here's an opportunity for leadership!
 - We'd be happy to provide advice especially to new projects
   interested in adding federation support
#+END_NOTES

** Beyond federation?

#+REVEAL_HTML: <img style="height: 550px;" src="../../crowdfunding/static/goblin_force-postcard-cropped-scaled.png" /><br />

#+BEGIN_NOTES
Federation as server to server tech is great, obviously I am for it
(tear down those silos!)

Maybe we don't have to stop there.  What are some things we can work
with or even learn from?
#+END_NOTES

*** The demonization and confinement of peer to peer

#+REVEAL_HTML: <img style="height: 550px;" src="../static/The_Pirate_Bay_logo.png" />

File sharing isn't evil, but peer to peer isn't just for file sharing
either...

#+BEGIN_NOTES
 - Let's talk about peer to peer!
 - Peer to peer is /not just file sharing!/
 - But it's mostly associated with it...
 - Hysteria around copyright violations lead to demonization
   of peer to peer in in mid-2000s
 - If P2P hadn't been demonized and more (not-just-file-sharing)
   work had been done, maybe we wouldn't be in the terrible place we
   are today.
#+END_NOTES

*** New ideas in peer to peer

#+REVEAL_HTML: <img style="height: 300px;" src="../static/Bitcoin_logo.svg" />

#+BEGIN_NOTES
 - We have new experiments in peer to peer, decentralized tech like
   bitcoin and other cryptocurrency
 - Could interest in expanding the role of peer to peer happen?
#+END_NOTES

*** Federation over peer to peer?

#+REVEAL_HTML: <img style="height: 300px; border: none;" src="../static/telehash.png" />

#+BEGIN_NOTES
 - Some interesting projects, for example telehash is for sending json snippets
   over peer to peer
 - Well, how about sending activitystreams activities?
 - Federation standard made by the social working group, with some small tweaks,
   probably could be adapted for peer to peer tech
 - Doesn't mean dropping federated social web, but an area to explore for sure
 - GNU has GNUNet, could it play a role?  I don't know enough about it.
   But seems an interesting place for intersection.
#+END_NOTES

** Everything that's old is GNU again

#+REVEAL_HTML: <img style="height: 550px;" src="../static/mud.png" />

Older systems may provide good ideas for newer tech!  (The dreams of
the 90s are alive in federation???)

#+BEGIN_NOTES
There are a lot of old ideas worth exploring that are relevant to
federation...

 - The "actor model" message passing ~= federated activitystreams...
   could make concurrent, federated programming much easier!
 - Capability security research (See the "Rees thesis")
 - MUDs and MOOs?
#+END_NOTES

*** Actually a huge topic of research in the 1990s

#+REVEAL_HTML: <img style="height: 550px;" src="../static/desiderata.png" />

#+BEGIN_NOTES
 - Text adventure games break down commands into something that looks
   close to message passing commands
 - LambdaMOO game player's guide == "database manual"
 - MUD/MOOs as REPLs to collaborative programming environments
 - Actual papers from research labs planning out federated MUD/MOO systems!
   (Solving super similar problems)

Sounds crazy?  But the point is, computer science history has a lot of
ideas to mine...
#+END_NOTES

** But back to MediaGoblin

#+REVEAL_HTML: <img style="height: 500px; border: none;" src="../static/pumpa_upload-from_pumpa2.png" />

 - 0.8.0 is right around the corner, supports client->server Pump API
 - server to server (federation) stuff is in progress, expected by
   next major release after

*** It's happening!

#+REVEAL_HTML: <img style="height: 600px; border: none;" src="../static/pumpa_upload-on_mediagoblin.png" />

*** Thanks to Jessica

#+REVEAL_HTML: <img style="height: 600px; border: none;" src="../static/jessica_tallon_screenshot.png" />

#+BEGIN_NOTES
 - Last year Evan Prodromou had introduced me to Jessica Tallon
   who had been working on a federation library named PyPump
 - I encouraged her to apply for a summer internship and she
   started working on federation through Outreach Program for Women
 - Doing such a good job that we brought her on (nearly) full time
 - Without Jessica's hard work and expertise, I'm not sure where we'd be
 - Give Jessica a hand!
#+END_NOTES

*** Thanks to you

# TODO: Put the screenshot of money raised?

#+REVEAL_HTML: <img style="height: 400px; border: none;" src="../../network_freedom/static/unlock_characters.png" />

#+BEGIN_NOTES
But this would not have been possible without your help!

Secretly that "second milestone" meant, hopefully bringing on a second
person.  And we were able to do that because of you.

Thank you to everyone who donated, you have made our work possible!
#+END_NOTES

* Thanks again (for listening!)

** Credits

 - Space colony art by NASA Ames Research Center, US Gov public domain
   http://settlement.arc.nasa.gov/70sArt/art.html
 - Pirate Bay logo is "Kopimi", taken from Wikipedia
 - MediaGoblin artwork where present mostly by me, CC BY-SA 4.0!
 - Bitcoin logo CC0 1.0 from bitcoin reference client
 - Telehash logo in the public domain, see telehash.org git repo

** Credits (fair use!)

Lots of pictures from Star Trek, which is proprietary and by Universal
Pictures, but fair use and stuff.

Sources:

: - http://asena42.blogspot.com/2011/08/live-long-and-prosper.html
: - http://tng.trekcore.com/gallery/albums/s1/1x17/homesoil004.jpg
: - https://i.imgur.com/5VCDtNw.jpg
: - https://startrektherewatch.files.wordpress.com/2014/11/skinofevil_hd_386.jpg?w=1180&h=885

Plus this thing by Rysiek:

: - http://events.ccc.de/congress/2014/wiki/Assembly:The_Federation

** Contact info / Questions?

# maybe something that isn't a shirtless kirk here

This talk CC BY-SA 4.0 International

 - *MediaGoblin:* http://mediagoblin.org/
 - *Me:* http://dustycloud.org/