Table of Contents
- 1. Introduction
- 2. A brief review of (Racket-flavored) Scheme
- 3. Goblins synchronize a Scheme-takeover
- 4. Goblins go asynchronous
- 4.1. Vats, at a high level
- 4.2. Making some vats
- 4.3. A friend
- 4.4. A friend that calls nearby friends
- 4.5. A friend that calls far away friends
- 4.6. Introducing <-
- 4.7. But what if there's errors?
- 4.8. "on" can return promises too
- 4.9. Promise chaining / pipelining is better
- 4.10. So why even have $?
- 4.11. But why not coroutines
- 5. Rage against the machines I haven't implemented yet
- 6. Rabbit holes (Goblins also welcome)
- 7. Thanks
1 Introduction
Goblins documentation: https://docs.racket-lang.org/goblins/
1.1 Ooh shiny
>> Show off time traveling Terminal Phase here <<
1.2 A minor troll (sorry Carl)
"Goblins was really easy to do because Sussman and Steele have already proven that scheme / the lambda calculus and actors are the same"
I'm sorry I'm sorry please don't kick me out of Friam
1.3 A Goblins recipe
- Defrost some E
- Prepare Scheme by carefully peeling off toxic "set!" rind. Prefer the W7 brand, but Racket will do and is more easily found in supermarkets.
- Place E and Scheme in blender, adding "become" as your actor state-management system
- Pulse for 30 seconds
- Pour into an iced glass, add a slice of lime, and enjoy
1.4 What does Goblins have?
- An "Actor" library (not a language) built on top of Racket / Scheme!
- (Could be ported to anything with lexical scope and weakmaps tho)
- Vat / "communicating event loops" style
- Quasi-functional object system!
- Transactional updates and time travel!
- Async programming with promises!
- Synchronous programming too! (But only in the same vat!)
1.5 What's not ready yet
- Fully distributed "machine to machine" stuff
- Ocap-safe module system (e-maker style)
- Ocap-safe macro system (DEFINITELY not ready, see cap-talk threads for current musings)
- drivers
1.6 Goblins architecture
Usable on two layers:
- Vats: One-turn-at-a-time event loops, set up ways to communicate with each other automatically.
- Actormaps: a bit more low level, vats technically wrap these. You can make and play with them directly. Great for games!
1.6.1 Abstraction nesting dolls
;; .--- "borrowed" from agoric??? ;; | ;; | .--- Chris: "How about I call this 'hive'?" ;; | | Ocap community: "We hate that" ;; | | Everyone else: "What's a 'vat' what a weird name" ;; | | ;; | | .--- Not a new idea, but new to expose to users? ;; | | | ;; | | | .--- Scheme already uses "-ref" everywhere ;; | | | | ;; | | | | .--- Actually a few kinds of these... ;; | | | | | most common is "mactor:local-actor" ;; | | | | | but there are others ;; | | | | | ;; | | | | | .--- Finally a term that is ;; | | | | | | unambiguous and well-understood ;; | | | | | | ;; V V V V V V (machine (vat (actormap {refr: (mactor object)})))
1.6.2 High level diagram
"Chris stop copying Mark's homework"
.----------------------------------. .----------------------. | Machine 1 | | Machine 2 | | ========= | | ========= | | | | | | .--------------. .---------. .-. .-. | | | Vat A | | Vat B | | \______| \_ .------------. | | | .---. | | .-. | .-| / | / | | Vat C | | | | (Alice)----------->(Bob)----' '-' '-' | | .---. | | | | '---' | | '-' | | | '--->(Carol) | | | | \ | '----^----' | | | '---' | | | | V | | | | | | | | | .----. | | .-. .-. | .------. | | | | (Alfred) | '-------/ |______/ |____---( Carlos ) | | | | '----' | \ | \ | | '------' | | | | | '-' '-' '------------' | | '--------------' | | | | | | | '----------------------------------' '----------------------'
Not shown here: vats wrapping actormaps, actormaps wrapping mactors
actor-id@vat-id
actor-id@vat-id[@machine-id[+hints]]
actor-id@machine-id
1.6.3 Notable things
- "Extremely minor" caveat: machines don't exist yet (coming this month?)
- Live references contain "vat connectors": compare with "eq?" to see if in main vat. Otherwise, vat connectors are procedures, use to talk to it
- Multiple kind of mactors but this is a rabbit hole
1.7 Shut up and show me the code
Ok, ok!
2 A brief review of (Racket-flavored) Scheme
2.1 Some simple procedures
;; Everyone's favorite (define (hello-world) "Hello, world!") ;; A procedure that returns an enclosed procedure (define (make-greeter greeter-name) (lambda (your-name) (format "Greetings ~a, my name is ~a!" your-name greeter-name))) ;; Assignment (define (make-counter) (define count 0) (lambda () (set! count (add1 count)) count))
2.2 Add methods?
- Method dispatch from procedures?
- Or procedures as objects with just-one-method?
We want to be schemey, so it's the former!
2.2.1 Doing it manually
;; Counter with very verbose manual dispatch (define (make-method-counter) (define count 0) (define (get-method method-name) ;; Pattern matching (match method-name ['get-count (lambda () count)] ['increment (lambda ([by-amount 1]) (set! count (+ count by-amount)))])) (define (dispatcher method-name . args) (define method (get-method method-name)) (apply method args)) dispatcher)
Yuck, but it works
2.2.2 Some sugar
(require goblins/actor-lib/methods) (define (make-method-counter2) (define count 0) (methods [(get-count) count] [(increment [by-amount 1]) (set! count (+ count by-amount))]))
MUCH easier to read!
Rabbithole: Extending yer methods
3 Goblins synchronize a Scheme-takeover
3.1 It's dangerous to go alone
Take an actormap:
(require goblins) (define am (make-actormap))
(NOTE: Most users won't use an actormap! They'll use a vat! Actormaps are lower level but I want to explain things to you all!)
3.2 Convert all previous examples to Goblins-style actors!
Topics to cover:
[X]Simple bcom-centric actors[X]Adding cells[X]Don't forget the methods one[X]Actormap context procedures[X]Oh hello there "spawn"[X]Oh hello there "$" (pronounced "dollar call"… why?)
[X]Different actormap procedures (for bootstrapping)[X]actormap-poke![X]actormap-peek[X]actormap-run![X]actormap-turn
[X]whactormaps and transactormaps[X]Transactions, aka "Errors? We forgot all about them!"
Rabbitholes:
3.3 Let's make our own cell
So simple I'll do it live!
(Writing that down didn't doom me or anything)
3.4 "bcom" is a sealer
And the syscaller / actormap has the unsealer, hidden in the mactor
3.5 Errors are no big deal
That is, if we use "bcom" instead of "set!"
3.6 Time check
Snapshotting time with snapshot-whactormap!
(Speaking of, this is Friam so is there even any time left by this point? We haven't even gotten to the asynchronous message passing stuff…)
4 Goblins go asynchronous
With our new friends, "<-" and "on"!
4.1 Vats, at a high level
Copying MarkM's homework again (this one Figure 14.2 from MarkM's dissertation), with minor adjustments:
.-----------------------.
|Internal Vat Schematics|
'======================='
stack heap
($) (actormap)
.-------.----------------------. -.
| | | |
| | .-. | |
| | (obj) .-. | |
| | '-' (obj) | |
| __ | '-' | |
| |__>* | .-. | |- actormap
| __ | (obj) | | territory
| |__>* | '-' | |
| __ | | |
| |__>* | | |
:-------'----------------------: -'
queue | __ __ __ | -.
(<-) | |__>* |__>* |__>* | |- event loop
'------------------------------' -' territory
4.2 Making some vats
It's still dangerous to go alone. But what if we do have friends, but in different places?
(require goblins) (define a-vat (make-vat)) (define b-vat (make-vat))
4.3 A friend
(define (^friend bcom my-name) (lambda (your-name) (format "Hello ~a, my name is ~a!" your-name my-name))) (define alice (a-vat 'spawn ^friend "Alice"))
Call Alice!
4.4 A friend that calls nearby friends
(define (^calls-friend bcom our-name) (lambda (friend) (define what-my-friend-said ($ friend our-name)) (displayln (format "<~a>: I called my friend, and they said:" our-name)) (displayln (format " \"~a\"" what-my-friend-said)))) (define archie (a-vat 'spawn ^calls-friend "Archie"))
Call Archie, who calls Alice!
4.5 A friend that calls far away friends
(define bob (b-vat 'spawn ^calls-friend "Bob"))
Call Bob who calls Alice!
… uhoh
4.6 Introducing <-
Let's prototype this:
(b-vat 'run (lambda () (<- alice "Brenda")))
Ok, that gives us a "promise"… but how to use it?
Aha… "on"! (Equivalent to E "when")
(b-vat 'run (lambda () (on (<- alice "Brenda") (lambda (alice-says) (displayln (format "Got from Alice: ~a" alice-says))))))
4.7 But what if there's errors?
Look at this fine coffee-sipping friend
(define (^this-is-fine bcom) (lambda () (error "Everything is on fire")))
We can catch that:
(b-vat 'run (lambda () (define its-fine-right (spawn ^this-is-fine)) (on (<- its-fine-right) (lambda (_val) (displayln (format "<narrator> And everything was indeed fine: ~a" val))) #:catch (lambda (err) (displayln (format "<narrator> But it wasn't fine at all: ~a" err))) #:finally (lambda () (displayln "And that was the story of whether things were fine.")))))
4.8 "on" can return promises too
(define ((^bakery bcom name) carb) (format "~a's signature ~a baking" name carb)) (define petite-oven-bakery (a-vat 'spawn ^bakery "The Petite Oven")) (a-vat 'run (lambda () (define smell-vow (on (<- petite-oven-bakery "croissants") (lambda (what-you-smell) (format "You smell ~a. Heavenly!!" what-you-smell)) #:promise? #t)) (on smell-vow displayln)))
4.9 Promise chaining / pipelining is better
"Machines grow faster and memories grow larger. But the speed of light is constant and New York is not getting any closer to Tokyo." – MarkM in Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control
4.9.1 A car factory
(define (^car-factory bcom company-name) (define ((^car bcom model color)) (format "*Vroom vroom!* You drive your ~a ~a ~a!" color company-name model)) (define (make-car model color) (spawn ^car model color)) make-car) (define fork-motors (a-vat 'spawn ^car-factory "Fork"))
fork-motors makes cars on a-vat, but what if we want to make and call from a remote vat?
But nested promises are terrible
(b-vat 'run (lambda () (on (<- fork-motors "Explorist" "blue") (lambda (our-car) (on (<- our-car) displayln)))))
4.9.2 Promise chaining is better
(b-vat 'run (lambda () (define car-vow (<- fork-motors "Explorist" "blue")) (define drive-noise-vow (<- car-vow)) (on drive-noise-vow displayln)))
4.9.3 Blah blah network latency blah blah
B => A => B => A => B
VS
B => A => B
4.9.4 Errors are also contagious
(define (^lessgood-car-factory bcom company-name) (define ((^car bcom model color)) (format "*Vroom vroom!* You drive your ~a ~a ~a!" color company-name model)) (define (make-car model color) (error "Your car exploded on the factory floor! Ooops!") (spawn ^car model color)) make-car) (define forked-motors (a-vat 'spawn ^lessgood-car-factory "Forked")) (b-vat 'run (lambda () (define car-vow (<- forked-motors "Exploder" "red")) (define drive-noise-vow (<- car-vow)) (on drive-noise-vow displayln #:catch (lambda (err) (displayln (format "Caught: ~a" err))))))
4.10 So why even have $?
- It's convenient when you can use it
- Necessary for transactional / atomic turns (ocap money)
4.11 But why not coroutines
- Goblins' predecessors (XUDD & 8sync, and first version of Goblins) all based on coroutines (under <<- operator, called "splitchronous communication")
- "Interleaving hazards" section of MarkM's dissertation reified many background worries
- async / await style "coroutine consent"
- possible, maybe ocap safe if human misleading
- haven't tried it yet
- is it worth it? I don't know. E style promise pipelining/chaining feels "nice enough" while still visible in its hazardousness
5 Rage against the machines I haven't implemented yet
5.1 What would addressing look like?
Sturdyref like <actor-id>@<machine-id>[<hints>] or something
Wait! Why not @vat-id?
5.2 Other CapTP blah blah
I mean, maybe we should just do a presentation when it's ready
6 Rabbit holes (Goblins also welcome)
6.1 Syscaller
When running low-level actormap-turn, actormap-spawn, a "syscaller" context is set up so that "<-" and "spawn" work without passing around a dynamic context
6.1.1 Wait this looks like dynamism how dare you
Why yes, there is a dynamic "syscaller" parameter inside the actor context…
Hidden deep in goblins/core.rkt:
;;; Syscaller internals ;;; =================== ;; NEVER export this. That would break our security paradigm. (define current-syscaller (make-parameter #f))
6.1.2 A play starring me and the ocap police
Me: Officer, I can explain!
Officer: WHY DO WE HAVE BOTH ACTORMAP-SPAWN AND SPAWN WHAT'S GOING ON HERE!
Me: That's… that's just for bootstrapping! There's no dynamism inside the actor context, I promise! It's just to make it so that "spawn" and "<-" don't require passing around the syscaller!
Officer: Don't you know that dynamism is ambient authority and references are the only form of security?
Me: No wait! Just think of it like a new language layer! It's like E!
Officer: Didn't you say this was a library and not a language?
Me: I… we can add a language layer! It's technically possible! It's Racket! They love making languages!
Officer: I've seen enough. Take 'em away boys!
The police officer rolls out a yellow tape around the crime scene which says: ABSTRACTION BARRIER: DO NOT CROSS
6.1.3 Alternate design path: pasing around a syscaller
(IF YOU SEE THIS MESSAGE YOU ARE TOO DEEP, ESCAPE THE RABBITHOLE NOW)
- Okay I guess if you have to (SwingSet does this I think)
- It's ugly though (sorry not sorry)
- Now we have two choices:
- Syscaller plus hidden transactormap mutation (not a big design change)
- Make syscaller a monad!
- Now with extra ugly
- Also this turns out to be insecure for some requirements!
- Paying for a newspaper, reading it, and then pretending like you never paid
- How about we just say "Goblins has an implicit monad"???
- Yeah! I like it! Something something "that's why time travel works!"
- It's arguably true and saying it confuses people too much for anyone to argue with me!
- If we paper over this with a language layer even less people would question it!
6.2 Mactors
(struct mactor ()) (struct mactor:local mactor ()) ;; TODO: Will this ever exist? It might only ever be a symlink. (struct mactor:remote mactor (vat-connid)) ;;; Resolved things ;;; --------------- ;; once a local refr, always a local refr. (struct mactor:local-actor mactor:local (handler become-unsealer become?)) ;; Once encased, always encased. ;; TODO: Maybe we don't need mactors for this. Maybe anything that's ;; not a mactor is basically "encased"? But having an official ;; mactor type gives us a clearer answer I guess. (struct mactor:encased mactor (val)) (struct mactor:remote-actor mactor:remote ()) (struct mactor:symlink mactor (link-to-refr)) ;; Once broken, always broken. (struct mactor:broken mactor (problem)) ;;; Eventual things ;;; --------------- (struct mactor:local-promise mactor:local (listeners resolver-unsealer resolver-tm?)) (struct mactor:remote-promise mactor:remote ())
6.3 Don't actors need to be able to know their own id?
- Not usually.
- But they can! Lexical scope to the rescue!
- Promise handlers reduce this need a lot… (have we gotten to "on" yet? How behind on time are we???)
6.4 Extending yer methods
[ ]Extending procedures[ ]Extending actors[ ]Racket-style mixins as a natural outgrowth[ ]Selfish actors
6.5 What about Goblins and Agoric/SwingSet/Jessie
"Yeah, what about it buddy???"
- I dunno? I have a hard time being motivated writing any non-lisps and I have a comfortable toolkit here
- Also I started this before SwingSet was announced and thought I had interesting ideas I was exploring and wanted to Get It Out There
- Obviously the Agoric folks Actually Know Things and I'm just this excited nerd on the internet so probably they know better than I do about basically everything imposter syndrome goes here
- Yes Jessie is probably the most scheme-like thing that exists in JS and is the best way of reaching people until WASM opens things up for every language (and yes I know that's a ways off, GC proposal, delimited continuations proposal, WASI, etc)
- Maybe Goblins should be ported to Jessie? I'd like to do it though I want to get the inter-machine stuff working yet. I am trying to figure out if I can work it into my schedule
- Let's collaborate on the CapTP thingamabob!
- Yes actually that sounds great
- Though I will note that I'm very intimidated by IBC
- Overcome'able?
6.6 Want to look at some code?
- Goblins internals
- Terminal Phase
7 Thanks
You all pretty much know this space better than I do I think
I hope you enjoyed this and I've contributed some ideas of interest
Thank you for your time! Seriously!