Talk outline

presentation/talk-outline.org

@@ -0,0 +1,316 @@
+#lang racket/base
+(require "../os2.rkt")
+(require "../os2-tcp.rkt")
+
+#|
+
+Outline ideas:
+ - the core of the idea is pubsub and presence. conversational
+   context - not raw messaging and actors.
+ - (frequently first-order-)functional programming of networked systems.
+    - similar to a generalisation of big-bang
+    - similar advantages obtain
+ - show code. Stuff that looks nice and illustrates the features of os2.
+    - transitions
+    - then the actions (spawn, roles, presence, messaging)
+    - then ground-vm
+    - then nested-vm
+    - then at-meta-level
+
+Prepare for whiteboard + showing code. Light on the slides, if any.
+
+Recent work in Network Application Architecture.
+
+---------------------------------------------------------------------------
+
+Desperately needs a name - "os-big-bang", "os2"
+
+Intro:
+
+Lots of systems, both formal models and practical implementations,
+work with messaging between actors or actor-like entities. Notably
+Erlang and the π-calculus.
+
+In these systems, actors/processes have conversations with one
+another, but the programming languages themselves don't talk about
+conversations or patterns of interaction larger than a single exchange
+of a message from a single sender to a single receiver.
+
+  Whiteboard
+  First point: receivers = 1 i.e. point-to-point/unicast
+  Second point: granularity = single message
+
+By contrast, lots of the communicating systems we want to model have
+more interesting conversational patterns.
+
+  HTTP: request/response; streaming; authentication; sessions
+  TCP: bidirectional; ongoing stream; connected
+  Bittorrent: multiparty; cooperative; unordered
+  Mailing lists: pub/sub; subscription management
+
+So generally we can say that communication is instead
+
+  Whiteboard
+  receivers = many i.e. multicast/broadcast/M-of-N
+  granularity = session/conversation/connection
+
+The research I've been doing has been examining the idea of a
+conversational context: the envelope within which a group of
+communicating actors/processes work together to achieve some shared
+task.
+
+There's a rich area of work dealing with describing the patterns of
+message exchange themselves, for example session types and behavioural
+contracts, but I'm looking at it from a different angle:
+
+ - how do actors interested in communicating find each other?
+     Whiteboard
+       Discovery
+       Synchronisation
+ - how does a conversation start?
+       Resource allocation
+ - how are resources managed during a conversation?
+       Acknowledgement
+       Nacks
+       Flow control
+ - how do participants know when a conversation is over?
+       Error/crash/exit signalling
+       Resource release
+ - how do conversations fit in to the underlying network system?
+       Routing
+ - what are the roles involved in various kinds of conversation?
+       Sender/Receiver
+       Scribe? Archivist? Statistician? Auditor?
+
+To experiment with this I've come up with a model operating system,
+and used it to implement a DNS caching proxy and an SSH server.
+
+The model operating system is based on multicast/pubsub, and also
+includes a construct for working with conversational context. I've
+been calling it "presence", since it's a generalisation of an idea
+from the XMPP/Jabber world, but you could equally well call it
+"interest" or "subscription".
+
+It's pure functional and event-based - very reminiscent of big-bang,
+and so has similar characteristics, including processes usually having
+first-order state - and recursive, meaning that instances of the
+kernel can run as user processes within another kernel instance.
+
+Each process within an instance of the system has private state, and a
+collection of active endpoints. Again the terminology here is a bit
+loose, and while I usually think of endpoints as subscriptions or
+event sources, they also make sense as TCP-like half-connections or
+more loosely as representing participation in a particular (set of)
+conversation(s).
+
+  Whiteboard
+
+  Process = ∃State . State × [Endpoint]
+    where
+      Endpoint = Topic × Role × InterestType
+                   × PresenceHandler
+                   × AbsenceHandler
+                   × MessageHandler
+
+      PresenceHandler = Topic           → State → Transition
+      AbsenceHandler  = Topic × Reason  → State → Transition
+      MessageHandler  = Message         → State → Transition
+
+  Transition = State × [Action]
+
+  Topic = notional set of Messages
+  Role = 'Subscriber + 'Publisher + ...
+  InterestType = 'Participant + 'Observer + 'SuperObserver
+
+The kernel runs a process in response to some external event, and it
+expects back a new process state and a list of actions that the
+process wishes the kernel to perform on its behalf.
+
+  Process = ∃State . State × [Endpoint]
+    where (...)
+      Action = 'Yield (State → Transition)
+             + 'AtMetaLevel Preaction
+             + Preaction
+      Preaction = 'AddRole Endpoint
+                + 'DeleteRole Endpoint
+                + 'SendMessage Message
+                + 'Spawn (∃State . State × [Action])
+
+  VMState = Process × [Action]
+
+  runVM :: VMState → VMState × [Action]
+
+  nestedVM :: (∃State . State × [Action]) → (VMState × [Action])
+  nestedVM bootProcess = ...
+
+  groundVM :: (∃State . State × [Action]) → 0
+  groundVM bootProcess = ...
+
+---------------------------------------------------------------------------
+|#
+
+
+
+
+
+
+
+
+
+
+
+(define (main)
+  (ground-vm
+   (transition 'none
+     (spawn tcp-driver)
+     (spawn listener))))
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+(define listener
+  (let ((local-addr (tcp-listener 5999)))
+    (transition 'no-state
+      (role 'inbound-handler
+	  (topic-subscriber (tcp-channel (wild) local-addr (wild))
+			    #:monitor? #t)
+	#:state state
+	#:topic t
+	#:on-presence
+	  (match t
+	    [(topic 'publisher (tcp-channel remote-addr (== local-addr) _) #f)
+	     (transition state
+	       (spawn (connection-handler local-addr remote-addr)))])))))
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+(define (connection-handler local-addr remote-addr)
+  (transition 'no-state
+    (send-feedback (tcp-channel remote-addr local-addr (tcp-mode 'lines)))
+    (send-feedback (tcp-channel remote-addr local-addr (tcp-credit 1)))
+    (role 'echoer
+	(topic-subscriber (tcp-channel remote-addr local-addr (wild)))
+      #:state state
+      #:on-absence (transition state
+		     (kill))
+      [(tcp-channel _ _ (? bytes? line))
+       (define reply (bytes-append #"You said: " line #"\n"))
+       (transition state
+	 (send-feedback (tcp-channel remote-addr local-addr (tcp-credit 1)))
+	 (send-message (tcp-channel local-addr remote-addr reply)))]
+      [(tcp-channel _ _ (? eof-object?))
+       (transition state
+	 (kill))])))
+
+
+
+
+
+
+#|
+---------------------------------------------------------------------------
+
+Messages include their topics: topics are after all just sets of messages
+ - This means ground events are (cons/c evt/c any/c)
+ - This is the set of all possible responses to sync'ing on that event!
+ - I've scraped by with this so far, but because sync'ing has
+   side-effects, I'll eventually need better names for event requests
+   and responsibility transfer.
+
+Contracts
+ - for process state (implemented)
+ - for messages across the bus at each level
+ - between processes / within conversations
+
+Functional pub/sub
+
+Motivate nested VM model
+ - Helper subprograms that have internal communication actions should
+   not have those actions manifest in their interface: what would be
+   stateful/side-effecting subroutine calls in Scheme should retain
+   that RPC-like request/response flavour in our system.
+ - Either the states of a subprogram and its callers should remain
+   separate, or some powerful help should be provided by the system to
+   make sure they are combined in a clean, manageable,
+   arbitrarily-composable fashion.
+ - The system should make it possible to invoke a subprogram without
+   needing to be aware of whether and how that subprogram in turn
+   chooses to invoke sub-subprograms.
+
+Motivate roles vs Erlang-style single mailbox
+ - The system should make it possible for programs that would
+   otherwise block on some external operation to remain responsive to
+   their normal inputs while waiting for the external operation to
+   complete.
+
+Can represent the "natural" structure of SSH in terms of nested VMs
+  Whiteboard
+  Diagram of 14 Oct 2011 from my research journal
+
+Ground-level events and a ground-level VM bottom out the recursion
+
+Examples
+  HTTP Load Balancer
+  Bank Teller
+
+Weaknesses
+ - The Wart
+ - Glitching
+    - There’s maybe a kind of continuousness (in the calculus sense)
+      at play here: well-behaved presence apps don’t glitch in their
+      presence. Sets of interests smoothly evolve without transient
+      drops of interest potentially leading to confusion on the part
+      of the peer.
+ - Handoff of responsibility
+    - the manager can take responsibility, spawn, and then drop when
+      it sees the child is up or crashed; or
+    - the manager can spawn without taking, and blip on crash; or
+    - the manager can spawn and then treat like an established service
+      instance
+ - Querying the routing table
+    - e.g. for sending SSH channel opens: is the channel type
+      supported? querying the routing table would let us find out
+      without explicitly sending a message
+
+Not just roles and sets, but *participation levels*
+ - participant
+ - observer/monitor
+ - "super"-observer/monitor
+
+This got me thinking: the presence/messaging system I’ve been building
+looks like a hybrid between synchronous and asynchronous messaging. Is
+it fair to say you can use the presence mechanism to synchronise, and
+then the messaging mechanism to communicate?
+
+Rendezvous via presence: this makes temporal dependencies between
+processes during system startup resolve themselves automatically. No
+more races at boot time!
+
+Exceptions and errors are propagated via loss of presence.
+ - Erlang's process links and monitors can be seen as a special case
+   of this mechanism.
+ - Erlang/OTP-like supervisors fit naturally into this picture
+
+|#