XUDD: An Async / Actor Model System for Python

1.4.1 Actors

1.4.2 Hives (actor queues)

<Elrond> BTW: Letting the hive do much of the "transporty" things has the    
         advantage of:                                                 
<Elrond> - No need to convert to json, if everything is local.
<Elrond> - No need to dns lookup uuids, if to and from a relocal objects.
<Elrond> - No need to assign a complex message-id, if it's local. Just use 
           something that is hash()able.                                 

1.4.3 Messaging between actors

1. Message basics
   

   Messages, when sent across the wire, are simple. They're just json documents, with the following fields supported:

   • from: the id of the actor that sent this message
   • to: a list of identifier(s), probably another actor's id (we might have some special "dns-like-lookup" actors later for special things)
   • message-id: this message
   • in-reply-to: the message this is replying to
   • body: a json document full of

   (They should be json-format as in termss of what fields anyway. They may be serialized when sent across the wire as msgpack or something… that's not important for now.)

   Maybe you're running a very simple text game which has two characters in a room… one fires a shot at another.

   {"from": "fd309f44-fb28-441d-b228-c9f35c520a79",
    "to": ["3c108d16-2acb-4a54-9398-dc8747fb4928"],
    "message-id": "bbb1d4c5-8565-4e17-a5cf-c94bc228d2f3",
    "directive": "attack",
    "body": {
      "type": "gunshot",
      "damage": 3,
      "accuracy": 40}}
   

   The recipient does some calculations regarding its own ability to dodg, determines that it is indeed hit, and lowers its hit points. It sends a message back to the other user informing that it was indeed damaged for 3 hit points, and is now at only 32 hit points.

   {"from": "3c108d16-2acb-4a54-9398-dc8747fb4928",
    "to": ["fd309f44-fb28-441d-b228-c9f35c520a79"],
    "message-id": "82f85f6a-a0b7-4e28-8e73-1e83e9ba3984",
    "in-reply-to": "bbb1d4c5-8565-4e17-a5cf-c94bc228d2f3",
    "directive": "attack_result",
    "body": {
      "success": True,
      "damage": 3,
      "hit-points-left": 32}}
   

   (We've ignore the mechanics of informing a room, etc… it might be a better pattern to pass the message through the room itself if your game works that way, that way the room can negotiate if the character is still there. But this is just a simple example!)

   For convenience, and since messages always follow this pattern, messages are wrapped in a Message object that also contains ActorProxy objects for each of the ids in from/to/message-id, etc.

2. Receiving and handling a message
   

   Actors operate whenever they receive a message. It's up to the actor to decide how to handle a message, but all actors must provide a receive_message() method.

   The default actor comes with a basic receive_message() method, but you can override it.

   By default, you define on an actor a set of message_handlers. Messages are routed depending on the directive.

   Let's give an example with a "vote counting" robot. This robot is super simple: it takes ballots and counts them until it's told to not accept ballots any more.

   class VotingMachine(Actor):
       def __init__(self):
           self.message_routing = {
               "enter_ballot": self.enter_ballot,
               "election_over": self.election_over}
   
           self.tally_sheet = {}
           self.election_running = True
   
       def enter_ballot(self, message):
           """
           Receive a ballot and increase the votes for the candidate by one.
           """
           if self.election_running:
               candidate = message.body['candidate']
               self.tally_sheet['candidate'] = self.tally_sheet.get(
                   message.body['candidate'], 0) + 1
   
       def election_over(self, message):
           """
           End the election!
           """
           self.election_running = False
   

3. Sending messages
   

   There are two ways to submit messages to an actor's hive.

   1. Fire and forget
      

      Basically, for a message that you don't need to "return" to in the same function, it should be possible to fire off a message to another actor and just continue on your way. For example, maybe we want to tell the voter from the above example whether or not their vote was accepted or rejected.

      Notice that we're adding a new self.hive.send_message call:

      class VotingMachine(Actor):
          def __init__(self):
              self.message_routing = {
                  "enter_ballot": self.enter_ballot,
                  "election_over": self.election_over}
      
              self.tally_sheet = {}
              self.election_running = True
      
          def enter_ballot(self, message):
              """
              Receive a ballot and increase the votes for the candidate by one.
              """
              if self.election_running:
                  candidate = message.body['candidate']
                  self.tally_sheet['candidate'] = self.tally_sheet.get(
                      message.body['candidate'], 0) + 1
      
                  # Inform the user that their ballot was accepted
                  self.hive.send_message(
                      to=message.from,
                      directive="inform_vote_result",
                      in_reply_to=message.id,
                      body={
                          result="accepted"})
              else:
                  # Inform the user that their ballot was rejected
                  self.hive.send_message(
                      to=message.from,
                      directive="inform_vote_result",
                      in_reply_to=message.id,
                      body={
                          result="rejected",
                          reason="The election is over."})
      
          def election_over(self, message):
              """
              End the election!
              """
              self.election_running = False
      

      With the above change, we send a message to the voter about whether or not their vote was accepted. But that's all we really care to do… once our votingmachine has processed the ballot and replied to the voter, they consider their job done.

      Another example of this might be something like a web application that needs to send off some task… say it got a video and it needs it transcoded. The web application doesn't need to sit around and wait for the video to finish transcoding before it sends a response to the user who submitted the video, so it fires off a task Celery-style to some worker actor, who will handle the rest of the transcoding.

   2. Yielding and waiting
      

      What happens if you send a message and you need to wait back on a reply? You could handle state on your actor and break things apart into a state machine with different functions depending on what part of the state of code you're writing. But shattering your functionality into a bunch of tiny functions can result in code that is really hard to follow.

      Enter python's "yield" statement. Yield does something fairly cool… it suspends a function mid-execution until it is woken up again (potentially with some result). This means we can fire a message off and our code will halt, but will wake up again and will continue executing once it has a response.

      (Note, this is a built-in functionality with the default Actor, but not all actors must implement things this way.)

      Say our voter is going to submit their ballot. They want to make sure they get their vote in… if they do, they'll stop feeling so anxious about voting and feel content. But if their vote isn't accepted for whatever reason (they've been hearing about problems with these machines…) they'll complain to the government and become indignant.

      Let's see what our Voter looks like then.

      class Voter(Actor):
          def __init__(self):
              self.message_routing = {
                  "go_vote": self.go_vote}
      
              # Our voter is anxious by default!
              self.mood = "anxious"
      
          def go_vote(self, message):
              """
              Our voter is reminded to go vote after polls open.
              """
              polling_place = message.body["voting_machine_id"]
      
              voting_result = yield self.wait_on_message(
                  to=polling_place,
                  directive="enter_ballot",
                  body={
                      "candidate": "Wesley Prelvis"})
      
              if voting_result["result"] == "accepted":
                  self.mood = "content"
      
              else:
                  self.hive.send_message(
                      to=message.body["municipal_id"],
                      directive="formal_complaint",
                      body={
                          "type": "voting problems",
                          "text": (
                              "Your stupid voting machine gave me this error: "
                              "'%s'") % voting_result["reason"]})
      
                  self.mood = "indignant"
      

      Our function pauses mid-execution at tye "yield" and waits for a response. Despite this all being one method, there's no blocking while waiting for that response, yet this is all written as a single, coherent method!

      What happens in the background is that this actor has a registry of messages that are "waiting" on a response. It tacks on a message id and waits for a reply. The next time the handle_message() method sees an incoming message with this id, it wakes up this method (suspended at that yield) and passes the incoming message to it.

      TODO:

      • document how to set timeouts!
      • better error handling? Or maybe not in this example to keep it simple ;)

1.4.4 Initial boot-up

1.4.5 Spawning an actor

1. Spawning at init
   
2. Spawning off another actor
   

1.4.6 Inter-hive communication

1. Inter-hive representatives
   
2. Actor lookup
   

1.4.7 Read-only actor properties

1.4.8 Subscribing to an actor's "events"

1.4.9 Serializing actors

1.4.10 Load balancing

1.4.11 "Safe passage" of local data

1.4.12 Supporting old applications

1.4.13 "Virtual" hives

1.4.14 Actor shutdown

1.4.15 Other ideas

1. Functional registry actors
   
2. Faster-ids
   

   import datetime
   
   def fast_message_id(origin_actor_id):
       dt = datetime.datetime.now()
       return "%s%s%s%s%s%s%s%s" % (
           "existing-actor-id", dt.year, dt.month,
           dt.day, dt.hour, dt.minute, dt.second, dt.microsecond)
   

   Even better idea:

   <Elrond> paroneayea - You could also create one uuid-per-actor at "boot" and  
            then append a counter. That should be pretty fast.                   
   <paroneayea> Elrond: oh yeah                                                  
   <paroneayea> that's a good idea                                               
   <paroneayea> so a counter per actor?                                          
   <Elrond> Right.
   

   <Elrond> paroneayea - Saw it.                                          
   <Elrond> paroneayea - Still could let the hive do the sending.
   <paroneayea> the hive will still do the sending
   <paroneayea> but the actor itself has to register the id               
   <paroneayea> so it knows what suspended coroutine to wake back up
   <Elrond> def self.wait_on_message():                                   
   <Elrond>   message_id = self.hive.send_message(...)
   <Elrond>   self.wait_for_message_with_id(message_id)
   <Elrond> Yeah.
   <Elrond> So that's why hive.send_message could return the message_id.
   <Elrond> So why not let hive.send_message return a message_id?
   <paroneayea> oh                                                        
   <Elrond> And let actor.wait_on_message take note of it?
   <paroneayea> yeah I suppose it could!
   <paroneayea> good point
   <Elrond> I am not saying, that this is best way since sliced bread though!   
            Just pointing out another option.                             
   

   <Elrond> *gna* That one is very tricky:
            http://29a.ch/2009/2/20/atomic-get-and-increment-in-python
   <Elrond> paroneayea - You might want to look at this ^^... If you want to
            implement the message-id_by_counter.
   

1.5.1 Error handling

1.5.2 Debugging

1.5.3 Clean shutdowns

1.5.4 How to "lock" actors? An actor should only do one task at a time.

3.1.1 Braindump

3.1.2 DONE Test out zeromq

3.2.1 TODO embed XMPP actor

3.3.1 DONE Add docstrings->docs support

3.3.2 TODO intro.rst

3.3.3 TODO quickstart.rst

3.3.4 TODO core.rst

3.7.1 TODO Allowing messages to have multiple recipients in to:

3.7.2 TODO Handling "collecting" this in yield

3.8.1 TODO Twisted

1. TODO Re-try twisted tutorial
   
2. TODO Check out twisted coroutines
   

3.8.2 TODO Watch Guido's talk on his async stuff

3.8.3 TODO Look at whatever that coroutine network one is

3.8.4 TODO Look at Go

3.9.1 TODO abcxy_talk

1. TODO Break this down into the core ideas
   
2. TODO Re-code
   

3.9.2 TODO noisyab

1. TODO Break this down into the core ideas
   
2. TODO Re-code