diff --git a/Concepts.html b/Concepts.html index 9deb4a9..2990f2b 100644 --- a/Concepts.html +++ b/Concepts.html @@ -1,10 +1,10 @@ -1 Concepts
On this page:
1.1 What is a process, what are event handlers?
1.2 What is a VM?
1.3 Endpoints:   Subscription and Advertisement
1.4 Messages and Topics
1.5 Presence
1.6 Nesting, relaying, and levels of discourse
5.3.4.10

1 Concepts

Marketplace integrates ideas from both distributed systems and +1 Concepts

5.3.4.10

1 Concepts

Marketplace integrates ideas from both distributed systems and virtualized operating system designs to obtain an architecture of nested virtual machines (VMs). Each nested layer is equipped with its own publish/subscribe network that also propagates presence information about the (dis)appearance of services.

Throughout this manual, diagrams such as the following will illustrate -various process structures:

image

Rectangular boxes represent VMs. The processes running within each VM +various process structures:

image

Rectangular boxes represent VMs. The processes running within each VM are placed atop its box. The narrow rectangular strip at the top of each VM’s box represents the network connecting all the VM’s processes to each other; it will frequently contain a short description of the @@ -24,8 +24,8 @@ handlers is called with the process’s current state and a description of the event. The handler is expected to return an updated state value and a collection of actions for the containing VM to perform. An event handler, then, has the following approximate type:

State × EventState × (Listof Action)

Event handlers are registered with the VM by creating endpoints -using the endpoint macro (described in Actions) or -the low-level add-endpoint structure (described in +using the endpoint macro (described in Creating endpoints) or +the low-level add-endpoint structure (described in Endpoints and Messages).

Events, passed to event handlers, describe the results of actions from the outside world, neighbouring processes in the VM, or the VM itself. They are implemented as structs. See @@ -33,8 +33,8 @@ the VM itself. They are implemented as Actions, passed back to the VM by event handlers, describe actions the process wishes to perform. See Actions for the possible actions a process can take.

Note that the result of an event handler function is actually a -Transition structure; the actual Typed Racket type of event -handlers is TrapK, defined in Handler Functions.

1.2 What is a VM?

Virtual Machines (VMs) are simply a collection +Transition structure; the actual Typed Racket type of event +handlers is TrapK, defined in Handler Functions.

1.2 What is a VM?

Virtual Machines (VMs) are simply a collection of processes, plus a shared medium of communication that the contained processes use to communicate with each other. VMs offer access to both their own internal network as well as to the external @@ -51,15 +51,15 @@ a good starting point is to google for in some protocol. They pair a description of the process’s role in a conversation with an event handler that responds to events relating to that role.

A role describes the role some process is playing in a -conversation. Concretely, roles are represented by Role +conversation. Concretely, roles are represented by Role structures. A role can be used by the currently-running process to describe some role it wishes to play, or can be carried in some -EndpointEvent to describe the role some peer process -is playing in a conversation.

Roles have three parts:

  • An orientation (type Orientation) describes +EndpointEvent to describe the role some peer process +is playing in a conversation.

    Roles have three parts:

    • An orientation (type Orientation) describes whether this role is concerned primarily with producing or consuming messages.

    • A topic is a pattern over messages. Topics perform double duty: they both scope conversations and filter -incoming messages. More on topics below.

    • An interest-type (type InterestType) +incoming messages. More on topics below.

    • An interest-type (type InterestType) determines whether the endpoint playing the given role is genuinely a participant in matching conversations or is simply observing the real participants. See Creating endpoints for more on @@ -68,12 +68,12 @@ value for which #:transparent structures, or any non-object structure for which prop:struct-map can be defined.

      As mentioned above, topics are simply patterns over messages. They are represented as normal data structures with embedded wildcards. -Use ? or (wild) to construct a wildcard. For +Use ? or (wild) to construct a wildcard. For example, given the following definition,

      (struct chat-message (speaker text) #:transparent)

      we can not only create instances that might be used with -send-message,

      (chat-message "Tony" "Hello World!")

      but also create topic patterns using ?. For example, this -pattern matches anything said by "Tony":

      (chat-message "Tony" ?)

      This pattern matches chat-messages sent by anyone saying "Hello":

      (chat-message ? "Hello")

      And finally, this pattern matches any chat-message at all:

      (chat-message ? ?)

      Patterns can be nested. For instance, given the above definition of +send-message,

      (chat-message "Tony" "Hello World!")

      but also create topic patterns using ?. For example, this +pattern matches anything said by "Tony":

      (chat-message "Tony" ?)

      This pattern matches chat-messages sent by anyone saying "Hello":

      (chat-message ? "Hello")

      And finally, this pattern matches any chat-message at all:

      (chat-message ? ?)

      Patterns can be nested. For instance, given the above definition of chat-message, the following pattern matches any chat message -greeting anybody at all:

      (struct greeting (target) #:transparent)
      (chat-message ? (greeting ?))

      1.5 Presence

      Presence (respectively its opposite, absence) is +greeting anybody at all:

      (struct greeting (target) #:transparent)
      (chat-message ? (greeting ?))

      1.5 Presence

      Presence (respectively its opposite, absence) is an indication that a matching conversational partner exists (resp. no longer exists) in the network. Presence can be used to synchronize conversations, setting up a conversational context before messages are @@ -85,10 +85,10 @@ in.

      The system derives presence information from the set of active pub/sub subscription and advertisement endpoints a process has created. Creating a new endpoint with a topic pattern that matches some other process’s endpoint and an orientation opposite to the other -process’s endpoint causes presence-events to be sent to both +process’s endpoint causes presence-events to be sent to both endpoints, informing them of the presence of the other. When a process -crashes, or an endpoint is withdrawn with delete-endpoint, a -corresponding absence-event is sent to the remaining +crashes, or an endpoint is withdrawn with delete-endpoint, a +corresponding absence-event is sent to the remaining endpoint.

      1.6 Nesting, relaying, and levels of discourse

      Because VMs can be nested, and each VM has an IPC network of its own for the use of its processes, information sometimes needs to be relayed from a VM’s external network to its internal network and vice @@ -97,7 +97,7 @@ may be quite different in syntax and meaning from those sent across the same VM’s external network: consider the case of the TCP chat server, which employs a nested VM to separate out TCP-related messages from higher-level, application-specific chat -messages:

      image

      Each VM’s network corresponds to a distinct level of discourse. +messages:

      image

      Each VM’s network corresponds to a distinct level of discourse. The nesting of VMs is then roughly analogous to the layering of network protocol stacks. For example (and purely hypothetically!) the -TCP-IP/HTTP/Webapp stack could perhaps be represented as

      image

 
\ No newline at end of file +TCP-IP/HTTP/Webapp stack could perhaps be represented as

image

 
\ No newline at end of file diff --git a/Drivers.html b/Drivers.html index 10c67a1..d0bbd64 100644 --- a/Drivers.html +++ b/Drivers.html @@ -1,45 +1,45 @@ -4 Drivers
On this page:
4.1 event-relay
event-relay
4.2 tcp-bare
tcp-driver
4.2.1 TCP channels
tcp-channel
Tcp  Sub  Packet
4.2.2 TCP addresses
Tcp  Address
tcp-address
tcp-handle
tcp-listener
4.2.3 Opening an outbound connection
4.2.4 Accepting inbound connections
4.2.5 Receiving data
4.2.6 Sending data
4.3 timer (typed and untyped)
4.4 udp (typed and untyped)
5.3.4.10

4 Drivers

4.1 event-relay

procedure

(event-relay self-id)  Spawn

  self-id : Symbol
Lets processes in some nested-vm interact with the outside -world using ground-vm-level event-based subscriptions.

Returns a spawn which starts an event-relay process with +4 Drivers
5.3.4.10

4 Drivers

4.1 event-relay

procedure

(event-relay self-id)  Spawn

  self-id : Symbol
Lets processes in some nested-vm interact with the outside +world using ground-vm-level event-based subscriptions.

Returns a spawn which starts an event-relay process with debug-name `(event-relay ,self-id).
The relay process observes subscriptions matching the topic-pattern -(cons (? evt?) _), and when one appears, constructs an -analogous one using at-meta-level to connect to the next VM +(cons (? evt?) _), and when one appears, constructs an +analogous one using at-meta-level to connect to the next VM down the stack. Messages from the meta-level will be relayed up to the current level. When the subscription disappears, the relay withdraws the subscription at the meta-level as well.

4.2 tcp-bare

This module is only available for use by untyped Racket processes. It is included by default in programs using #lang marketplace; see Using #lang marketplace and friends for information on other language -variants.

procedure

(tcp-driver)  Spawn

Returns a spawn action which starts a TCP driver. The TCP +variants.

procedure

(tcp-driver)  Spawn

Returns a spawn action which starts a TCP driver. The TCP driver should run either directly in a ground VM, or in a nested VM -with a running event-relay.

4.2.1 TCP channels

struct

(struct tcp-channel (source destination subpacket)
  #:prefab)
  source : TcpAddress
  destination : TcpAddress
  subpacket : TcpSubPacket
A TCP channel represents a section of a unidirectional TCP flow +with a running event-relay.

value

tcp : Spawn

A pre-made spawn action equivalent to (tcp-driver).

4.2.1 TCP channels

struct

(struct tcp-channel (source destination subpacket)
  #:prefab)
  source : TcpAddress
  destination : TcpAddress
  subpacket : TcpSubPacket
A TCP channel represents a section of a unidirectional TCP flow appearing on our local "subnet" of the full TCP network, complete with source, destination and subpacket. Each TCP connection has two such flows: one inbound (remote-to-local) bytestream, and one outbound -(local-to-remote) bytestream.

type

TcpSubPacket : (or/c eof-object? bytes?)

Packets carried by tcp-channel structures are either +(local-to-remote) bytestream.

type

TcpSubPacket : (or/c eof-object? bytes?)

Packets carried by tcp-channel structures are either end-of-file objects or raw binary data represented as Racket byte -vectors.

4.2.2 TCP addresses

type

TcpAddress : (or/c tcp-address? tcp-handle? tcp-listener?)

A TCP address describes one end of a TCP connection. It can be either

  • a tcp-address, representing a remote socket;

  • a tcp-handle, representing a local socket on a kernel-assigned port; or

  • a tcp-listener, representing a local socket on a user-assigned port.

struct

(struct tcp-address (host port)
  #:prefab)
  host : string?
  port : (integer-in 0 65535)
Describes a remote half-connection. The host part is to be a +vectors.

4.2.2 TCP addresses

type

TcpAddress : (or/c tcp-address? tcp-handle? tcp-listener?)

A TCP address describes one end of a TCP connection. It can be either

  • a tcp-address, representing a remote socket;

  • a tcp-handle, representing a local socket on a kernel-assigned port; or

  • a tcp-listener, representing a local socket on a user-assigned port.

struct

(struct tcp-address (host port)
  #:prefab)
  host : string?
  port : (integer-in 0 65535)
Describes a remote half-connection. The host part is to be a string containing either a hostname (e.g. "localhost") or an -ASCII representation of an IP address (e.g. "127.0.0.1").

struct

(struct tcp-handle (id)
  #:prefab)
  id : any/c
Describes a local half-connection with a kernel-assigned port number. +ASCII representation of an IP address (e.g. "127.0.0.1").

struct

(struct tcp-handle (id)
  #:prefab)
  id : any/c
Describes a local half-connection with a kernel-assigned port number. The port number is not directly accessible; the id is used as a local name for whichever underlying port number ends up being used.

The id must be chosen carefully: it is scoped to the local VM, i.e. shared between processes in that VM, so processes must make sure not to accidentally clash in handle ID selection. They are also used in TcpChannel to mean a specific instance of a TCP connection, so if you are likely to want to reconnect individual -flows, use different values for id.

struct

(struct tcp-listener (port)
  #:prefab)
  port : (integer-in 0 65535)
Describes a local half-connection with a user-assigned port number. -Use this to describe server sockets.

4.2.3 Opening an outbound connection

Choose a tcp-handle, and then create endpoints as follows:

(let ((local (tcp-handle 'some-unique-value))
      (remote (tcp-address "the.remote.host.example.com" 5999)))
  (transition/no-state
   (endpoint #:publisher (tcp-channel local remote ?))
   (endpoint #:subscriber (tcp-channel remote local ?)
             [(tcp-channel _ _ (? eof-object?))
              ; Handle a received end-of-file object
              (transition ...)]
             [(tcp-channel _ _ (? bytes? data))
              ; Handle received data
              (transition ...)])))

The TCP driver will automatically create an outbound connection in +flows, use different values for id.

struct

(struct tcp-listener (port)
  #:prefab)
  port : (integer-in 0 65535)
Describes a local half-connection with a user-assigned port number. +Use this to describe server sockets.

4.2.3 Opening an outbound connection

Choose a tcp-handle, and then create endpoints as follows:

(let ((local (tcp-handle 'some-unique-value))
      (remote (tcp-address "the.remote.host.example.com" 5999)))
  (transition/no-state
   (endpoint #:publisher (tcp-channel local remote ?))
   (endpoint #:subscriber (tcp-channel remote local ?)
             [(tcp-channel _ _ (? eof-object?))
              ; Handle a received end-of-file object
              (transition ...)]
             [(tcp-channel _ _ (? bytes? data))
              ; Handle received data
              (transition ...)])))

The TCP driver will automatically create an outbound connection in response to the presence of the endpoints. When the endpoints are deleted (or the process exits), the TCP driver will notice the absence and will close the underlying TCP socket.

For a complete example, see TCP chat client.

4.2.4 Accepting inbound connections

Choose a port number, and then create an observer endpoint as -follows:

(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) #:observer
          #:conversation (tcp-channel them us _)
          #:on-presence (spawn #:child (chat-session them us)))

The use of #:observer here indicates that this endpoint isn’t +follows:

(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) #:observer
          #:conversation (tcp-channel them us _)
          #:on-presence (spawn #:child (chat-session them us)))

The use of #:observer here indicates that this endpoint isn’t actually interested in exchanging any TCP data; instead, it is monitoring demand for such exchanges. The TCP driver uses a rare #:everything endpoint to monitor the presence of #:observers, and creates listening TCP server sockets in response. When a connection comes in, the TCP driver spawns a manager process which offers regular #:participant endpoints for -communicating on the newly-arrived socket.

To illustrate the code for handling a newly-arrived connection,

(define (chat-session them us)
  (transition/no-state
   (endpoint #:subscriber (tcp-channel them us ?)
             #:on-absence (quit)
             [(tcp-channel _ _ (? bytes? data))
              ; Handle incoming data
              (transition ...)])))
4.2.5 Receiving data

TCP-related messages will be of the form

(tcp-channel remote-address local-address subpacket)

where the subpacket is either eof or a -bytes?.

4.2.6 Sending data

Send data with

(send-message (tcp-channel local-address remote-address subpacket))

where, as for receiving data, the subpacket is either +communicating on the newly-arrived socket.

To illustrate the code for handling a newly-arrived connection,

(define (chat-session them us)
  (transition/no-state
   (endpoint #:subscriber (tcp-channel them us ?)
             #:on-absence (quit)
             [(tcp-channel _ _ (? bytes? data))
              ; Handle incoming data
              (transition ...)])))
4.2.5 Receiving data

TCP-related messages will be of the form

(tcp-channel remote-address local-address subpacket)

where the subpacket is either eof or a +bytes?.

4.2.6 Sending data

Send data with

(send-message (tcp-channel local-address remote-address subpacket))

where, as for receiving data, the subpacket is either eof or a bytes?.

4.3 timer (typed and untyped)

For examples of the use of the timer driver, see uses of set-timer and timer-expired in the diff --git a/Examples.html b/Examples.html index 2a7c75a..dc2bb0a 100644 --- a/Examples.html +++ b/Examples.html @@ -1,6 +1,6 @@ -6 Examples

6 Examples

6.1 TCP echo server

Here is a complete Marketplace program:

"examples/echo-paper.rkt"

#lang marketplace
 
(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?)
          #:conversation (tcp-channel from to _)
          #:on-presence (spawn #:child (echoer from to)))
 
(define (echoer from to)
(transition stateless
(endpoint #:subscriber (tcp-channel from to ?)
          #:on-absence (quit)
          [(tcp-channel _ _ data)
           (send-message (tcp-channel to from data))])))

The top-level endpoint action subscribes to TCP connections -arriving on port 5999, and spawns a fresh process in response to +6 Examples

5.3.4.10

6 Examples

6.1 TCP echo server

Here is a complete Marketplace program:

"examples/echo-paper.rkt"

#lang marketplace
 
(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?)
          #:conversation (tcp-channel from to _)
          #:on-presence (spawn #:child (echoer from to)))
 
(define (echoer from to)
(transition stateless
(endpoint #:subscriber (tcp-channel from to ?)
          #:on-absence (quit)
          [(tcp-channel _ _ data)
           (send-message (tcp-channel to from data))])))

The top-level endpoint action subscribes to TCP connections +arriving on port 5999, and spawns a fresh process in response to each (#:on-presence). The topic of conversation (#:conversation) associated with the newly-present subscription is analyzed to give the remote @@ -9,11 +9,11 @@ passed to the echoer function to give the initial ac the corresponding process. Here, the process is stateless, using the special constant stateless as its state.

Each connection’s process creates an endpoint subscribing to data arriving on its particular connection, using from and to -passed in from the top-level endpoint. When data arrives, it is -echoed back to the remote peer using send-message. Presence +passed in from the top-level endpoint. When data arrives, it is +echoed back to the remote peer using send-message. Presence manages disconnection; when the remote peer closes the TCP connection, -the #:on-absence handler in echoer issues a quit +the #:on-absence handler in echoer issues a quit action, terminating the connection’s process. The heart of our system is the interface between a process and its containing VM. Our implementation instantiates this interface as a collection of Typed -Racket programs.

6.2 TCP chat server

"examples/chat-paper.rkt"

#lang marketplace
 
(nested-vm
 (at-meta-level
  (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) #:observer
            #:conversation (tcp-channel them us _)
            #:on-presence (spawn #:child (chat-session them us)))))
 
(define (chat-session them us)
(define user (gensym 'user))
(transition stateless
            (listen-to-user user them us)
            (speak-to-user user them us)))
 
(define (listen-to-user user them us)
(list
 (endpoint #:publisher `(,user says ,?))
 (at-meta-level
  (endpoint #:subscriber (tcp-channel them us ?)
            #:on-absence (quit)
            [(tcp-channel _ _ (? bytes? text))
             (send-message `(,user says ,text))]))))
 
(define (speak-to-user user them us)
(define (say fmt . args)
(at-meta-level (send-message (tcp-channel us them (apply format fmt args)))))
(define (announce who did-what)
(unless (equal? who user) (say "~s ~s.~n" who did-what)))
(list
 (say "You are ~s.~n" user)
 (at-meta-level
  (endpoint #:publisher (tcp-channel us them ?)))
(endpoint #:subscriber `(,? says ,?)
          #:conversation `(,who says ,_)
          #:on-presence (announce who 'arrived)
          #:on-absence  (announce who 'departed)
          [`(,who says ,what) (say "~a: ~a" who what)])))

6.3 TCP chat client

"examples/chat-client.rkt"

#lang marketplace
(require racket/port)
 
(spawn #:debug-name 'console-output-driver
       #:child
       (transition/no-state
        (endpoint #:subscriber (list 'console-output ?)
                  [(list 'console-output item)
                   (begin (printf "~a" item)
                          (void))])))
 
(spawn #:debug-name 'console-input-driver
       #:child
       (transition/no-state
        (endpoint #:publisher (list 'console-input ?)
                  #:name 'input-relay
                  #:on-absence
                  (list (send-message (list 'console-output "Connection terminated.\n"))
                        (quit)))
(endpoint #:subscriber (cons (read-line-evt (current-input-port) 'any) ?)
          [(cons _ (? eof-object?))
           (list (send-message (list 'console-output "Terminating on local EOF.\n"))
                 (delete-endpoint 'input-relay))]
[(cons _ (? string? line))
 (send-message (list 'console-input line))])))
 
(spawn #:debug-name 'outbound-connection
       #:child
       (let ((local (tcp-handle 'outbound))
             (remote (tcp-address "localhost" 5999)))
(transition/no-state
 (endpoint #:subscriber (list 'console-input ?)
           #:on-absence (quit)
           [(list 'console-input line)
            (list (send-message (list 'console-output (format "> ~a \n" line)))
                  (send-message (tcp-channel local remote (string-append line "\n"))))])
(endpoint #:publisher (tcp-channel local remote ?))
(endpoint #:subscriber (tcp-channel remote local ?)
          #:on-absence (quit)
          [(tcp-channel _ _ (? eof-object?))
           (quit)]
[(tcp-channel _ _ data)
 (list (send-message (list 'console-output (format "< ~a" data)))
       (void))]))))
 
\ No newline at end of file +Racket programs.

6.2 TCP chat server

"examples/chat-paper.rkt"

#lang marketplace
 
(nested-vm
 (at-meta-level
  (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) #:observer
            #:conversation (tcp-channel them us _)
            #:on-presence (spawn #:child (chat-session them us)))))
 
(define (chat-session them us)
(define user (gensym 'user))
(transition stateless
            (listen-to-user user them us)
            (speak-to-user user them us)))
 
(define (listen-to-user user them us)
(list
 (endpoint #:publisher `(,user says ,?))
 (at-meta-level
  (endpoint #:subscriber (tcp-channel them us ?)
            #:on-absence (quit)
            [(tcp-channel _ _ (? bytes? text))
             (send-message `(,user says ,text))]))))
 
(define (speak-to-user user them us)
(define (say fmt . args)
(at-meta-level (send-message (tcp-channel us them (apply format fmt args)))))
(define (announce who did-what)
(unless (equal? who user) (say "~s ~s.~n" who did-what)))
(list
 (say "You are ~s.~n" user)
 (at-meta-level
  (endpoint #:publisher (tcp-channel us them ?)))
(endpoint #:subscriber `(,? says ,?)
          #:conversation `(,who says ,_)
          #:on-presence (announce who 'arrived)
          #:on-absence  (announce who 'departed)
          [`(,who says ,what) (say "~a: ~a" who what)])))

6.3 TCP chat client

"examples/chat-client.rkt"

#lang marketplace
(require racket/port)
 
(spawn #:debug-name 'console-output-driver
       #:child
       (transition/no-state
        (endpoint #:subscriber (list 'console-output ?)
                  [(list 'console-output item)
                   (begin (printf "~a" item)
                          (void))])))
 
(spawn #:debug-name 'console-input-driver
       #:child
       (transition/no-state
        (endpoint #:publisher (list 'console-input ?)
                  #:name 'input-relay
                  #:on-absence
                  (list (send-message (list 'console-output "Connection terminated.\n"))
                        (quit)))
(endpoint #:subscriber (cons (read-line-evt (current-input-port) 'any) ?)
          [(cons _ (? eof-object?))
           (list (send-message (list 'console-output "Terminating on local EOF.\n"))
                 (delete-endpoint 'input-relay))]
[(cons _ (? string? line))
 (send-message (list 'console-input line))])))
 
(spawn #:debug-name 'outbound-connection
       #:child
       (let ((local (tcp-handle 'outbound))
             (remote (tcp-address "localhost" 5999)))
(transition/no-state
 (endpoint #:subscriber (list 'console-input ?)
           #:on-absence (quit)
           [(list 'console-input line)
            (list (send-message (list 'console-output (format "> ~a \n" line)))
                  (send-message (tcp-channel local remote (string-append line "\n"))))])
(endpoint #:publisher (tcp-channel local remote ?))
(endpoint #:subscriber (tcp-channel remote local ?)
          #:on-absence (quit)
          [(tcp-channel _ _ (? eof-object?))
           (quit)]
[(tcp-channel _ _ data)
 (list (send-message (list 'console-output (format "< ~a" data)))
       (void))]))))
 
\ No newline at end of file diff --git a/Management_and_Monitoring.html b/Management_and_Monitoring.html index a13422f..de41a41 100644 --- a/Management_and_Monitoring.html +++ b/Management_and_Monitoring.html @@ -1,21 +1,21 @@ -5 Management and Monitoring
5.3.4.10

5 Management and Monitoring

5.1 generic-spy

procedure

(generic-spy label)  Spawn

  label : Any
Returns a spawn action that, when executed, creates a process -with a #:subscriber endpoint listening for every -message. Each EndpointEvent received by the endpoint is +5 Management and Monitoring
5.3.4.10

5 Management and Monitoring

5.1 generic-spy

procedure

(generic-spy label)  Spawn

  label : Any
Returns a spawn action that, when executed, creates a process +with a #:subscriber endpoint listening for every +message. Each EndpointEvent received by the endpoint is printed to the current output port. Using this process gives a crude -trace of activity within a VM: presence-events and -absence-events (of #:publishers) are logged, as is +trace of activity within a VM: presence-events and +absence-events (of #:publishers) are logged, as is each 'publisher message sent to the VM’s network.

5.2 logging (MATRIX_LOG)

environment variable

MATRIX_LOG

Set the MATRIX_LOG environment variable to "debug", "info", "warning", "error" or "fatal" (i.e. any of Racket’s log-level?s) to enable output of log messages at that level and higher.

If MATRIX_LOG is not defined in the environment, no log -output will be produced.

syntax

(matrix-log level format-str arg ...)

 
level = expr
     
format-str = expr
     
arg = expr
Analogous to Racket’s core log-message, but uses -matrix-root-logger instead of the system logger. The +output will be produced.

syntax

(matrix-log level format-str arg ...)

 
level = expr
     
format-str = expr
     
arg = expr
Analogous to Racket’s core log-message, but uses +matrix-root-logger instead of the system logger. The level expression must evaluate to a level symbol (see log-level?), and format-str must evaluate to a -format string for use with format.

value

matrix-root-logger : logger?

The root logger for marketplace logging.

5.3 debugger (experimental)

procedure

(debug p)  Spawn

  p : Spawn
Translates a spawn action to another spawn action which wraps +format string for use with format.

The root logger for marketplace logging.

5.3 debugger (experimental)

procedure

(debug p)  Spawn

  p : Spawn
Translates a spawn action to another spawn action which wraps the to-be-spawned process in a debugging interface. Executing the resulting action will not only create a process in the executing VM, but will also open a debugger GUI.

N.B.: The debugger is experimental and likely to change quite quickly and unpredictably.
See the file "examples/debug-chat.rkt" for an example of the -use of debug.

 
\ No newline at end of file +use of debug.

 
\ No newline at end of file diff --git a/high-level-interface.html b/high-level-interface.html index e645848..c2e1c21 100644 --- a/high-level-interface.html +++ b/high-level-interface.html @@ -1,5 +1,5 @@ -2 High-level interface
5.3.4.10

2 High-level interface

This high-level interface between a VM and a process is analogous to +2 High-level interface

5.3.4.10

2 High-level interface

This high-level interface between a VM and a process is analogous to the C library interface of a Unix-like operating system. The Low-level interface corresponds to the system call interface of a Unix-like operating system.

2.1 Using #lang marketplace and friends

 #lang marketplace

Programs written for Marketplace differ from normal Racket modules @@ -13,65 +13,65 @@ be a variation for Typed Racket, and languages providing greater support for flow control, responsibility transfer, and other networking concepts. For now, Typed Racket programs must be written as #lang typed/racket programs using (require marketplace) -and ground-vm: explicitly.

2.2 Using Marketplace as a library

 (require marketplace/sugar-untyped)
 (require marketplace/sugar-typed)

Instead of using Racket’s #lang feature, ordinary Racket programs +and ground-vm: explicitly.

2.2 Using Marketplace as a library

 (require marketplace/sugar-untyped)
 (require marketplace/sugar-typed)

Instead of using Racket’s #lang feature, ordinary Racket programs can use Marketplace features by requiring Marketplace modules -directly.

Such programs need to use ground-vm/ground-vm: to +directly.

Such programs need to use ground-vm/ground-vm: to start the ground-level VM explicitly. They also need to explicitly start any drivers they need; for example, the file -"examples/echo-plain.rkt" uses ground-vm along with -tcp and an initial endpoint action:

(ground-vm
  tcp
  (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?)
            #:conversation (tcp-channel from to _)
            #:on-presence (spawn #:child (echoer from to))))

syntax

(ground-vm maybe-boot-pid-binding maybe-initial-state initial-action ...)

syntax

(ground-vm: maybe-boot-pid-binding maybe-typed-initial-state initial-action ...)

 
maybe-boot-pid-binding = 
  | #:boot-pid id
     
maybe-initial-state = 
  | #:initial-state expr
     
maybe-typed-initial-state = 
  | #:initial-state expr : type
     
initial-action = expr
Starts the ground VM, in untyped and typed programs, respectively. If +"examples/echo-plain.rkt" uses ground-vm along with +tcp and an initial endpoint action:

(ground-vm
  tcp
  (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?)
            #:conversation (tcp-channel from to _)
            #:on-presence (spawn #:child (echoer from to))))

syntax

(ground-vm maybe-boot-pid-binding maybe-initial-state initial-action ...)

syntax

(ground-vm: maybe-boot-pid-binding maybe-typed-initial-state initial-action ...)

 
maybe-boot-pid-binding = 
  | #:boot-pid id
     
maybe-initial-state = 
  | #:initial-state expr
     
maybe-typed-initial-state = 
  | #:initial-state expr : type
     
initial-action = expr
Starts the ground VM, in untyped and typed programs, respectively. If #:boot-pid is specified, the given identifier is bound within the form to the PID of the primordial process that performs the initial actions. If #:initial-state is specified (with a -type, for ground-vm:), it is used as the initial state for +type, for ground-vm:), it is used as the initial state for the primordial process; if it is not supplied, the primordial process is given (void) as its initial state (and Void as -its state type).

2.3 Constructing transitions

syntax

(transition new-state action-tree ...)

syntax

(transition: new-state : State action-tree ...)

syntax

(transition/no-state action-tree ...)

Each of these forms produces a Transition structure. The +its state type).

2.3 Constructing transitions

syntax

(transition new-state action-tree ...)

syntax

(transition: new-state : State action-tree ...)

syntax

(transition/no-state action-tree ...)

Each of these forms produces a Transition structure. The first is for untyped code, the second for typed code (where the mandatory State is the type of the transitioning process’s -private state), and the third for either.

Each action-tree must be an (ActionTree State).

It’s fine to include no action-trees, in which case the +private state), and the third for either.

Each action-tree must be an (ActionTree State).

It’s fine to include no action-trees, in which case the transition merely updates the state of the process without taking any -actions.

In the case of transition/no-state, the type Void +actions.

In the case of transition/no-state, the type Void and value (void) is used for the process state. -transition/no-state is useful for processes that are -stateless other than the implicit state of their endpoints.

struct

(struct transition (state actions)
  #:transparent)
  state : State
  actions : (ActionTree State)

type

Transition : (All (State) (transition State))

A transition structure. The transition-state field is the new +transition/no-state is useful for processes that are +stateless other than the implicit state of their endpoints.

struct

(struct transition (state actions)
  #:transparent)
  state : State
  actions : (ActionTree State)

type

Transition : (All (State) (transition State))

A transition structure. The transition-state field is the new private state the process will have after the transition is applied, -and the transition-actions are the actions that will be -performed by the VM in order to apply the transition.

type

ActionTree : (All (State) (Constreeof (Action State)))

type

Constreeof : (All (X) (Rec CT (U X (Pairof CT CT) False Void Null)))

An action-tree is a cons-tree of Actions. When +and the transition-actions are the actions that will be +performed by the VM in order to apply the transition.

type

ActionTree : (All (State) (Constreeof (Action State)))

type

Constreeof : (All (X) (Rec CT (U X (Pairof CT CT) False Void Null)))

An action-tree is a cons-tree of Actions. When performing actions, a VM will traverse an action-tree in left-to-right order.

'(), (void), and #f may also be present in action-trees: when the VM reaches such a value, it ignores it and continues with the next leaf in the tree.

For example, all of the following are valid action trees which will -send messages 1, 2 and 3 in that order:

(list (send-message 1)
      (send-message 2)
      (send-message 3))
(list (list (send-message 1))
      (cons (send-message 2) (cons '() (send-message 3))))
(cons (cons (send-message 1)
            (send-message 2))
      (list #f #f (send-message 3)))

Because #f and (void) are valid, ignored, members of +send messages 1, 2 and 3 in that order:

(list (send-message 1)
      (send-message 2)
      (send-message 3))
(list (list (send-message 1))
      (cons (send-message 2) (cons '() (send-message 3))))
(cons (cons (send-message 1)
            (send-message 2))
      (list #f #f (send-message 3)))

Because #f and (void) are valid, ignored, members of an action-tree, and and when can be used to selectively include actions in an action-tree:

(list (first-action)
      (when (condition?)
        (optional-action))
      (final-action))
(list (first-action)
      (and (condition?)
           (optional-action))
      (final-action))

Finally, these inert placeholders can be used to represent "no action -at all" in a transition:

(transition new-state) ; No action-trees at all
(transition new-state '())
(transition new-state (void))
(transition new-state #f)

procedure

(sequence-actions initial-transition    
  item ...)  (Transition State)
  initial-transition : (Transition State)
  item : 
(U (ActionTree State)
   (State -> (Transition State)))
Returns a transition formed from the initial-transition +at all" in a transition:

(transition new-state) ; No action-trees at all
(transition new-state '())
(transition new-state (void))
(transition new-state #f)

procedure

(sequence-actions initial-transition    
  item ...)  (Transition State)
  initial-transition : (Transition State)
  item : 
(U (ActionTree State)
   (State -> (Transition State)))
Returns a transition formed from the initial-transition extended with new actions, possibly updating its carried state. Each of the supplied items is examined: if it is an -ActionTree, it is appended to the pending transition’s +ActionTree, it is appended to the pending transition’s actions; if it is a procedure, it is called with the state of the pending transition, and is expected to return an updated -transition.

For example,

(sequence-actions (transition 'x
                    (send-message (list 'message 0)))
                  (send-message (list 'message 1))
                  (send-message (list 'message 2))
                  (lambda (old-state)
                    (transition (cons 'y old-state)
                      (send-message (list 'message 3))))
                  (send-message (list 'message 4)))

produces the equivalent of

(transition (cons 'y 'x)
  (send-message (list 'message 0))
  (send-message (list 'message 1))
  (send-message (list 'message 2))
  (send-message (list 'message 3))
  (send-message (list 'message 4)))

2.4 Creating endpoints

The primitive action that creates new endpoints is -add-endpoint, but because endpoints are the most flexible and +transition.

For example,

(sequence-actions (transition 'x
                    (send-message (list 'message 0)))
                  (send-message (list 'message 1))
                  (send-message (list 'message 2))
                  (lambda (old-state)
                    (transition (cons 'y old-state)
                      (send-message (list 'message 3))))
                  (send-message (list 'message 4)))

produces the equivalent of

(transition (cons 'y 'x)
  (send-message (list 'message 0))
  (send-message (list 'message 1))
  (send-message (list 'message 2))
  (send-message (list 'message 3))
  (send-message (list 'message 4)))

2.4 Creating endpoints

The primitive action that creates new endpoints is +add-endpoint, but because endpoints are the most flexible and complex point of interaction between a process and its VM, a DSL, -endpoint, streamlines endpoint setup.

syntax

(endpoint orientation topic maybe-interest-type
          maybe-let-name
          maybe-name
          maybe-state-pattern
          maybe-on-presence
          maybe-on-absence
          maybe-role-patterns
          maybe-reason-pattern
          maybe-message-handlers)

syntax

(endpoint: maybe-typed-state-pattern : State
           orientation topic maybe-interest-type
           maybe-let-name
           maybe-name
           maybe-on-presence
           maybe-on-absence
           maybe-role-patterns
           maybe-reason-pattern
           maybe-message-handlers)
 
maybe-typed-state-pattern = 
  | pattern
     
orientation = #:subscriber
  | #:publisher
     
topic = expr
     
maybe-interest-type = 
  | #:participant
  | #:observer
  | #:everything
     
maybe-let-name = 
  | #:let-name identifier
     
maybe-name = 
  | #:name expr
     
maybe-state-pattern = 
  | #:state pattern
     
maybe-on-presence = 
  | #:on-presence handler-expr
     
maybe-on-absence = 
  | #:on-absence handler-expr
     
maybe-role-patterns = 
  | #:role pattern
  | 
#:peer-orientation pattern
#:conversation pattern
#:peer-interest-type pattern
     
maybe-reason-pattern = 
  | #:reason pattern
     
maybe-message-handlers = 
  | message-handler ...
     
message-handler = [pattern handler-expr]
     
handler-expr = expr
Almost everything is optional in an endpoint. The only +endpoint, streamlines endpoint setup.

syntax

(endpoint orientation topic maybe-interest-type
          maybe-let-name
          maybe-name
          maybe-state-pattern
          maybe-on-presence
          maybe-on-absence
          maybe-role-patterns
          maybe-reason-pattern
          maybe-message-handlers)

syntax

(endpoint: maybe-typed-state-pattern : State
           orientation topic maybe-interest-type
           maybe-let-name
           maybe-name
           maybe-on-presence
           maybe-on-absence
           maybe-role-patterns
           maybe-reason-pattern
           maybe-message-handlers)
 
maybe-typed-state-pattern = 
  | pattern
     
orientation = #:subscriber
  | #:publisher
     
topic = expr
     
maybe-interest-type = 
  | #:participant
  | #:observer
  | #:everything
     
maybe-let-name = 
  | #:let-name identifier
     
maybe-name = 
  | #:name expr
     
maybe-state-pattern = 
  | #:state pattern
     
maybe-on-presence = 
  | #:on-presence handler-expr
     
maybe-on-absence = 
  | #:on-absence handler-expr
     
maybe-role-patterns = 
  | #:role pattern
  | 
#:peer-orientation pattern
#:conversation pattern
#:peer-interest-type pattern
     
maybe-reason-pattern = 
  | #:reason pattern
     
maybe-message-handlers = 
  | message-handler ...
     
message-handler = [pattern handler-expr]
     
handler-expr = expr
Almost everything is optional in an endpoint. The only mandatory parts are the orientation and the topic. For -endpoint:, the expected type of the process state must also -be supplied.

For example, a minimal endpoint subscribing to all messages would be:

(endpoint #:subscriber ?)

or in Typed Racket, for a process with Integer as its process -state type,

(endpoint: : Integer #:subscriber ?)

A minimal publishing endpoint would be:

(endpoint #:publisher ?)
(endpoint: : Integer #:publisher ?)

While topic patterns are ordinary Racket data with embedded ? +endpoint:, the expected type of the process state must also +be supplied.

For example, a minimal endpoint subscribing to all messages would be:

(endpoint #:subscriber ?)

or in Typed Racket, for a process with Integer as its process +state type,

(endpoint: : Integer #:subscriber ?)

A minimal publishing endpoint would be:

(endpoint #:publisher ?)
(endpoint: : Integer #:publisher ?)

While topic patterns are ordinary Racket data with embedded ? wildcards (see Messages and Topics), all the other patterns -in an endpoint are match-patterns. In particular -note that ? is a wildcard in a topic pattern, while +in an endpoint are match-patterns. In particular +note that ? is a wildcard in a topic pattern, while _ is a wildcard in a match-pattern.

2.4.1 Receiving messages

Supply one or more message-handler clauses to handle incoming message events (as distinct from presence- or absence-events).

The following endpoint subscribes to all messages, but only -handles some of them:

(endpoint #:subscriber ?
          ['ping (send-message 'pong)]
          ['hello (list (send-message 'goodbye)
                        (quit))])
2.4.2 Action-only vs. State updates

If #:state occurs in an endpoint, or the -maybe-typed-state-pattern occurs in an endpoint:, +handles some of them:

(endpoint #:subscriber ?
          ['ping (send-message 'pong)]
          ['hello (list (send-message 'goodbye)
                        (quit))])
2.4.2 Action-only vs. State updates

If #:state occurs in an endpoint, or the +maybe-typed-state-pattern occurs in an endpoint:, then all the handler-exprs in that endpoint are expected to return transition structures.

If not, however, the event handler expressions are expected to return -plain ActionTrees.

This way, simple endpoints that do not need to examine the process +plain ActionTrees.

This way, simple endpoints that do not need to examine the process state, and simply act in response to whichever event triggered them, can be written without the clutter of threading the process state -value through the code.

For example, a simple endpoint could be written either as

(endpoint #:subscriber 'ping
          ['ping (send-message 'pong)])

or, explicitly accessing the endpoint’s process’s state,

(endpoint #:subscriber 'ping
          #:state old-state
          ['ping (transition old-state
                   (send-message 'pong))])
2.4.3 Naming endpoints

Endpoint names can be used to update +value through the code.

For example, a simple endpoint could be written either as

(endpoint #:subscriber 'ping
          ['ping (send-message 'pong)])

or, explicitly accessing the endpoint’s process’s state,

(endpoint #:subscriber 'ping
          #:state old-state
          ['ping (transition old-state
                   (send-message 'pong))])
2.4.3 Naming endpoints

Endpoint names can be used to update or delete endpoints.

If #:name is supplied, the given value is used as the name of the endpoint. If not, a fresh name is created. (At present, gensym is used.)

If #:let-name is supplied, the given identifier is bound in @@ -81,8 +81,8 @@ and complementary orientations to the current endpoint. When such endpoints come and go, presence and absence events are generated in the current endpoint.

By default, no actions are taken on such events, but #:on-presence and #:on-absence override this -behaviour.

For example, say process A establishes the following endpoint:

(endpoint #:subscriber 'ping
          #:on-presence (send-message 'pinger-arrived)
          #:on-absence  (send-message 'pinger-departed)
          ['ping (send-message 'pong)])

Some time later, process B takes the following endpoint-establishing -action:

(endpoint #:publisher 'ping
          #:let-name ping-endpoint-name
          #:on-presence
          (list (endpoint #:subscriber 'pong
                          #:let-name pong-waiter-name
                          ['pong (list (delete-endpoint ping-endpoint-name)
                                       (delete-endpoint pong-waiter-name))])
                (send-message 'ping)))

The sequence of events will be:

  1. Process A’s #:on-presence handler will run, and the +behaviour.

    For example, say process A establishes the following endpoint:

    (endpoint #:subscriber 'ping
              #:on-presence (send-message 'pinger-arrived)
              #:on-absence  (send-message 'pinger-departed)
              ['ping (send-message 'pong)])

    Some time later, process B takes the following endpoint-establishing +action:

    (endpoint #:publisher 'ping
              #:let-name ping-endpoint-name
              #:on-presence
              (list (endpoint #:subscriber 'pong
                              #:let-name pong-waiter-name
                              ['pong (list (delete-endpoint ping-endpoint-name)
                                           (delete-endpoint pong-waiter-name))])
                    (send-message 'ping)))

    The sequence of events will be:

    1. Process A’s #:on-presence handler will run, and the 'pinger-arrived message will be sent. At the same time,In the current implementation, one happens before the other, but it is nondeterministic which is run first. process B’s @@ -95,8 +95,8 @@ the 'pinger-departed mes someone is listening for pings.

      This way, if process B starts before process A, then B will automatically wait until A is ready to receive ping requests before issuing any.

      2.4.5 Exit reasons

      If a #:reason pattern is supplied, then the exit reason -supplied to the delete-endpoint or quit action that -led to the absence-event is available to the endpoint’s +supplied to the delete-endpoint or quit action that +led to the absence-event is available to the endpoint’s #:on-absence handler expression.

      2.4.6 Updating endpoints

      If, when an endpoint is created, an existing endpoint with an equal? name is already present, then if the existing and to-be-added endpoints have exactly equal roles (meaning equal @@ -107,65 +107,65 @@ signals. A future release of this library will include better automatic support for avoiding such transients.

      2.4.7 Who am I talking to?

      If either #:role or any of #:peer-orientation, #:conversation, or #:peer-interest-type are supplied, the handler-exprs are given access to the role -carried in the EndpointEvent that triggered them.

      This role describes the intersection of interests between the +carried in the EndpointEvent that triggered them.

      This role describes the intersection of interests between the current endpoint and the peer endpoint, and so can proxy for the identity of the other party. It is in a sense a description of the scope of the current conversation.

      Using #:role allows a handler complete access to the -role structure in the triggering event. It is more common +role structure in the triggering event. It is more common however to simply use #:conversation to extract the -role-topic alone, since it is seldom necessary to examine -role-orientation (since it’s guaranteed to be complementary +role-topic alone, since it is seldom necessary to examine +role-orientation (since it’s guaranteed to be complementary to the orientation of the current endpoint) or -role-interest-type. If access to those parts is required, use -#:peer-orientation and #:peer-interest-type.

      2.5 Deleting endpoints

      procedure

      (delete-endpoint id [reason])  Action

        id : Any
        reason : Any = #f
      Use this action to delete a previously-added endpoint by name. The -delete-endpoint-id must be equal? to the -corresponding add-endpoint-pre-eid; when endpoint -was used to construct the endpoint to be deleted, the relevant name is -that bound by #:let-name or supplied to #:name. See +role-interest-type. If access to those parts is required, use +#:peer-orientation and #:peer-interest-type.

      2.5 Deleting endpoints

      procedure

      (delete-endpoint id [reason])  Action

        id : Any
        reason : Any = #f
      Use this action to delete a previously-added endpoint by name. The +id given must be equal? to the corresponding +add-endpoint-pre-eid; when endpoint was used to +construct the endpoint to be deleted, the relevant name is that bound +by #:let-name or supplied to #:name. See Naming endpoints.

      If reason is supplied, it is included in the corresponding -action, and made available in any resulting absence-events.

      2.6 Sending messages and feedback

      procedure

      (send-message body [orientation])  Action

        body : Any
        orientation : Orientation = 'publisher
      Constructs a message-sending action with the given orientation. +action, and made available in any resulting absence-events.

      2.6 Sending messages and feedback

      procedure

      (send-message body [orientation])  Action

        body : Any
        orientation : Orientation = 'publisher
      Constructs a message-sending action with the given orientation. Usually the correct orientation to use is 'publisher; it means that the sender of the message is acting in the "publisher" role. Use 'subscriber instead when acting in the "subscriber" -role, i.e. sending feedback.

      procedure

      (send-feedback body)  Action

        body : Any
      Equivalent to (send-message body 'subscriber).

      2.7 Creating processes

      syntax

      (spawn maybe-pid-binding maybe-debug-name maybe-parent-continuation
             #:child boot-expr)

      syntax

      (spawn: maybe-pid-binding maybe-debug-name typed-parent-continuation
              #:child : ChildStateType boot-expr)
       
      maybe-pid-binding = 
        | #:pid identifier
           
      maybe-debug-name = 
        | #:debug-name expr
           
      maybe-parent-continuation = 
        | #:parent k-expr
        | #:parent parent-state-pattern k-expr
           
      typed-parent-continuation = #:parent : ParentStateType
        | #:parent : ParentStateType k-expr
        | #:parent parent-state-pattern : ParentStateType k-expr
           
      k-expr = expr
           
      boot-expr = expr
      Action describing a new process to create. The boot-expr -should be an expression yielding a transition that contains +role, i.e. sending feedback.

      procedure

      (send-feedback body)  Action

        body : Any
      Equivalent to (send-message body 'subscriber).

      2.7 Creating processes

      syntax

      (spawn maybe-pid-binding maybe-debug-name maybe-parent-continuation
             #:child boot-expr)

      syntax

      (spawn: maybe-pid-binding maybe-debug-name typed-parent-continuation
              #:child : ChildStateType boot-expr)
       
      maybe-pid-binding = 
        | #:pid identifier
           
      maybe-debug-name = 
        | #:debug-name expr
           
      maybe-parent-continuation = 
        | #:parent k-expr
        | #:parent parent-state-pattern k-expr
           
      typed-parent-continuation = #:parent : ParentStateType
        | #:parent : ParentStateType k-expr
        | #:parent parent-state-pattern : ParentStateType k-expr
           
      k-expr = expr
           
      boot-expr = expr
      Action describing a new process to create. The boot-expr +should be an expression yielding a transition that contains the child process’s initial state and initial actions.

      If #:pid is supplied, the associated identifier is bound to the child process’s PID in both boot-expr and the parent’s k-expr.

      Any supplied #:debug-name will be used in VM debug output. See also logging (MATRIX_LOG).

      If #:parent is supplied, the associated k-expr will run in the parent process after the child process has been created. If the parent-state-pattern is also supplied, then -k-expr must return a Transition; otherwise, it must -return an ActionTree. Note that in Typed Racket, for type -system reasons, spawn: requires ParentStateType to -be supplied.

      2.8 Exiting and killing processes

      procedure

      (quit [who reason])  Action

        who : (Option PID) = #f
        reason : Any = #f
      Action causing the termination of a process. If who is +k-expr must return a Transition; otherwise, it must +return an ActionTree. Note that in Typed Racket, for type +system reasons, spawn: requires ParentStateType to +be supplied.

      2.8 Exiting and killing processes

      procedure

      (quit [who reason])  Action

        who : (Option PID) = #f
        reason : Any = #f
      Action causing the termination of a process. If who is omitted or #f, terminates the acting process; otherwise, terminates the peer process having who as its PID.

      If reason is supplied, it is included in the corresponding -action, and made available in any resulting absence-events.

      Terminating the current process is as simple as:

      (quit)

      When a process raises an exception that it does not catch, its +action, and made available in any resulting absence-events.

      Terminating the current process is as simple as:

      (quit)

      When a process raises an exception that it does not catch, its containing VM catches the exception and turns it into an implicit quit action. In that case, the reason will be the raised exception -itself.

      2.9 Cooperative scheduling

      syntax

      (yield maybe-state-pattern k-expr)

      syntax

      (yield: typed-state-pattern k-expr)

       
      maybe-state-pattern = 
        | #:state pattern
           
      typed-state-pattern = : State
        | pattern : State
           
      k-expr = expr
      Lets other processes in the system run for a step, returning to +itself.

      2.9 Cooperative scheduling

      syntax

      (yield maybe-state-pattern k-expr)

      syntax

      (yield: typed-state-pattern k-expr)

       
      maybe-state-pattern = 
        | #:state pattern
           
      typed-state-pattern = : State
        | pattern : State
           
      k-expr = expr
      Lets other processes in the system run for a step, returning to evaluate k-expr only after doing a complete round of the scheduler.

      If pattern is supplied, k-expr should evaluate to a -Transition; otherwise it should produce an ActionTree.

      2.10 Creating nested VMs

      syntax

      (nested-vm maybe-vm-pid-binding maybe-boot-pid-binding
                 maybe-initial-state
                 maybe-debug-name
                 boot-action-expr ...)

      syntax

      (nested-vm: : ParentStateType
                  maybe-vm-pid-binding maybe-boot-pid-binding
                  maybe-typed-initial-state
                  maybe-debug-name
                  boot-action-expr ...)
       
      maybe-vm-pid-binding = 
        | #:vm-pid identifier
           
      maybe-boot-pid-binding = 
        | #:boot-pid identifier
           
      maybe-initial-state = 
        | #:initial-state expr
           
      maybe-typed-initial-state = 
        | #:initial-state expr : StateType
           
      maybe-debug-name = 
        | #:debug-name expr
           
      boot-action-expr = expr
      Results in a spawn action that starts a nested VM. The +Transition; otherwise it should produce an ActionTree.

      2.10 Creating nested VMs

      syntax

      (nested-vm maybe-vm-pid-binding maybe-boot-pid-binding
                 maybe-initial-state
                 maybe-debug-name
                 boot-action-expr ...)

      syntax

      (nested-vm: : ParentStateType
                  maybe-vm-pid-binding maybe-boot-pid-binding
                  maybe-typed-initial-state
                  maybe-debug-name
                  boot-action-expr ...)
       
      maybe-vm-pid-binding = 
        | #:vm-pid identifier
           
      maybe-boot-pid-binding = 
        | #:boot-pid identifier
           
      maybe-initial-state = 
        | #:initial-state expr
           
      maybe-typed-initial-state = 
        | #:initial-state expr : StateType
           
      maybe-debug-name = 
        | #:debug-name expr
           
      boot-action-expr = expr
      Results in a spawn action that starts a nested VM. The primordial process in the new VM executes the boot-actions with the given initial state. (If no initial state is supplied, (void) is used.)

      If #:vm-pid is present, the corresponding identifier is bound in the boot-action expressions to the container-relative PID of the new VM itself. If #:boot-pid is present, however, the corresponding identifier is bound to the new-VM-relative PID of the -primordial process in the new VM.

      2.11 Relaying across layers

      syntax

      (at-meta-level: : StateType preaction ...)

      procedure

      (at-meta-level preaction ...)  (Action StateType)

        preaction : (PreAction State)
      Each VM gives its processes access to two distinct IPC facilities: the +primordial process in the new VM.

      2.11 Relaying across layers

      syntax

      (at-meta-level: : StateType preaction ...)

      procedure

      (at-meta-level preaction ...)  (Action StateType)

        preaction : (PreAction State)
      Each VM gives its processes access to two distinct IPC facilities: the internal one, provided for the VM’s processes to talk amongst themselves, and the external one, the network that the VM itself is a process within.

      Marketplace’s actions can apply to either of those two networks. By default, actions apply to the VM of the acting process directly, but -using at-meta-level (or at-meta-level: in typed +using at-meta-level (or at-meta-level: in typed code) to wrap an action level-shifts the action to make it -apply at the level of the acting process’s VM’s container instead.

      For example, wrapping an endpoint in at-meta-level +apply at the level of the acting process’s VM’s container instead.

      For example, wrapping an endpoint in at-meta-level adds a subscription to the VM’s container’s network. Instead of listening to sibling processes of the acting process, the new endpoint will listen to sibling processes of the acting process’s VM. In this -example, the primordial process in the nested-vm creates an -endpoint in the VM’s own network, the ground VM:

      (nested-vm
       (at-meta-level
        (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) ...)))

      In this example, a new process is spawned as a sibling of the -nested-vm rather than as a sibling of its primordial process:

      (nested-vm
       (at-meta-level
        (spawn #:child (transition/no-state (send-message 'hello-world)))))

      Compare to this example, which spawns a sibling of the -nested-vm’s primordial process:

      (nested-vm
       (spawn #:child (transition/no-state (send-message 'hello-world))))
       
      \ No newline at end of file +example, the primordial process in the nested-vm creates an +endpoint in the VM’s own network, the ground VM:

      (nested-vm
       (at-meta-level
        (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) ...)))

      In this example, a new process is spawned as a sibling of the +nested-vm rather than as a sibling of its primordial process:

      (nested-vm
       (at-meta-level
        (spawn #:child (transition/no-state (send-message 'hello-world)))))

      Compare to this example, which spawns a sibling of the +nested-vm’s primordial process:

      (nested-vm
       (spawn #:child (transition/no-state (send-message 'hello-world))))
       
      \ No newline at end of file diff --git a/index.html b/index.html index f5fee15..424ba0a 100644 --- a/index.html +++ b/index.html @@ -34,4 +34,4 @@ location of mutable state, failure detection and recovery, access control, I/O and user interface, debugging and profiling.

      Marketplace addresses these concerns with a small set of primitives chosen to make network programming in-the-small as flexible, scalable, manageable and securable as network programming in-the-large—and -vice versa.

          1 Concepts

            1.1 What is a process, what are event handlers?

            1.2 What is a VM?

            1.3 Endpoints: Subscription and Advertisement

            1.4 Messages and Topics

            1.5 Presence

            1.6 Nesting, relaying, and levels of discourse

          2 High-level interface

            2.1 Using #lang marketplace and friends

            2.2 Using Marketplace as a library

            2.3 Constructing transitions

            2.4 Creating endpoints

              2.4.1 Receiving messages

              2.4.2 Action-only vs. State updates

              2.4.3 Naming endpoints

              2.4.4 Handling presence and absence events

              2.4.5 Exit reasons

              2.4.6 Updating endpoints

              2.4.7 Who am I talking to?

            2.5 Deleting endpoints

            2.6 Sending messages and feedback

            2.7 Creating processes

            2.8 Exiting and killing processes

            2.9 Cooperative scheduling

            2.10 Creating nested VMs

            2.11 Relaying across layers

          3 Low-level interface

            3.1 Handler Functions

            3.2 Topics and Roles

            3.3 Endpoint Events

            3.4 Actions

              3.4.1 Endpoints and Messages

              3.4.2 Process Management

          4 Drivers

            4.1 event-relay

            4.2 tcp-bare

              4.2.1 TCP channels

              4.2.2 TCP addresses

              4.2.3 Opening an outbound connection

              4.2.4 Accepting inbound connections

              4.2.5 Receiving data

              4.2.6 Sending data

            4.3 timer (typed and untyped)

            4.4 udp (typed and untyped)

          5 Management and Monitoring

            5.1 generic-spy

            5.2 logging (MATRIX_LOG)

            5.3 debugger (experimental)

          6 Examples

            6.1 TCP echo server

            6.2 TCP chat server

            6.3 TCP chat client

       
      \ No newline at end of file +vice versa.

          1 Concepts

            1.1 What is a process, what are event handlers?

            1.2 What is a VM?

            1.3 Endpoints: Subscription and Advertisement

            1.4 Messages and Topics

            1.5 Presence

            1.6 Nesting, relaying, and levels of discourse

          2 High-level interface

            2.1 Using #lang marketplace and friends

            2.2 Using Marketplace as a library

            2.3 Constructing transitions

            2.4 Creating endpoints

              2.4.1 Receiving messages

              2.4.2 Action-only vs. State updates

              2.4.3 Naming endpoints

              2.4.4 Handling presence and absence events

              2.4.5 Exit reasons

              2.4.6 Updating endpoints

              2.4.7 Who am I talking to?

            2.5 Deleting endpoints

            2.6 Sending messages and feedback

            2.7 Creating processes

            2.8 Exiting and killing processes

            2.9 Cooperative scheduling

            2.10 Creating nested VMs

            2.11 Relaying across layers

          3 Low-level interface

            3.1 Handler Functions

            3.2 Messages, Topics and Roles

            3.3 Endpoint Events

            3.4 Actions

              3.4.1 Endpoints and Messages

              3.4.2 Process Management

          4 Drivers

            4.1 event-relay

            4.2 tcp-bare

              4.2.1 TCP channels

              4.2.2 TCP addresses

              4.2.3 Opening an outbound connection

              4.2.4 Accepting inbound connections

              4.2.5 Receiving data

              4.2.6 Sending data

            4.3 timer (typed and untyped)

            4.4 udp (typed and untyped)

          5 Management and Monitoring

            5.1 generic-spy

            5.2 logging (MATRIX_LOG)

            5.3 debugger (experimental)

          6 Examples

            6.1 TCP echo server

            6.2 TCP chat server

            6.3 TCP chat client

 
\ No newline at end of file diff --git a/low-level-interface.html b/low-level-interface.html index de1e9b0..bc33918 100644 --- a/low-level-interface.html +++ b/low-level-interface.html @@ -1,5 +1,5 @@ -3 Low-level interface
5.3.4.10

3 Low-level interface

 (require marketplace)

At its heart, the interface between each process and its +3 Low-level interface

5.3.4.10

3 Low-level interface

 (require marketplace)

At its heart, the interface between each process and its containing VM is based on handler functions exchanging event and action structures with the VM. Both events and actions are simple Racket structures.

This low-level interface between a VM and a process is analogous to @@ -7,24 +7,26 @@ the system call interface of a Unix-like High-level interface corresponds to the C library interface of a Unix-like operating system.

3.1 Handler Functions

Each handler function is always associated with a particular endpoint, registered with the VM via -endpoint/endpoint:/add-endpoint. A handler -function for a given process with state type State has type:

(EndpointEvent -> State -> (Transition State))

That is, given an EndpointEvent followed by the process’s -current state, the handler should reply with a Transition -containing a new process state and a collection of Actions.

type

Handler : (All (State) (TrapK State))

type

TrapK : (All (State) (EndpointEvent -> (InterruptK State)))

type

InterruptK : (All (State) (State -> (Transition State)))

Typed Racket types capturing various notions of handler function.

3.2 Topics and Roles

type

Topic : Any

As previously mentioned, topics are ordinary Racket values -which may have embedded wildcards.

value

? : Topic

Each time ? is used in an expression context, it produces a -fresh topic wildcard, suitable for use in a topic pattern.

struct

(struct role (orientation topic interest-type)
  #:prefab)
  orientation : Orientation
  topic : Topic
  interest-type : InterestType

type

Role : role

Roles are almost always constructed by the -endpoint/endpoint: macros or by the VM +endpoint/endpoint:/add-endpoint. A handler +function for a given process with state type State has type:

(EndpointEvent -> State -> (Transition State))

That is, given an EndpointEvent followed by the process’s +current state, the handler should reply with a Transition +containing a new process state and a collection of Actions.

type

Handler : (All (State) (TrapK State))

type

TrapK : (All (State) (EndpointEvent -> (InterruptK State)))

type

InterruptK : (All (State) (State -> (Transition State)))

Typed Racket types capturing various notions of handler function.

3.2 Messages, Topics and Roles

type

Message : Any

type

Topic : Any

As previously mentioned, messages are ordinary Racket values, +and topics are ordinary Racket values which may have embedded +wildcards.

procedure

(wild)  Topic

syntax

? : Topic

Each time ? (or (wild)) is used in an expression +context, it produces a fresh topic wildcard, suitable for use in a +topic pattern.

struct

(struct role (orientation topic interest-type)
  #:prefab)
  orientation : Orientation
  topic : Topic
  interest-type : InterestType

type

Role : role

Roles are almost always constructed by the +endpoint/endpoint: macros or by the VM implementations themselves. User programs generally only need to -destructure role instances.

A role describes the conversational role of a peer as seen by +destructure role instances.

A role describes the conversational role of a peer as seen by some process. For example, a subscriber to topic 'foo with interest-type 'participant might receive a presence -notification carrying the role

(role 'publisher 'foo 'participant)

Notice that the orientation of the role is the opposite of the -orientation of the endpoint.

type

Orientation : (U 'publisher 'subscriber)

Describes an endpoint’s orientation: will it be acting as a publisher +notification carrying the role

(role 'publisher 'foo 'participant)

Notice that the orientation of the role is the opposite of the +orientation of the endpoint.

type

Orientation : (U 'publisher 'subscriber)

Describes an endpoint’s orientation: will it be acting as a publisher of messages, or as a subscriber to messages? Publishers (orientation -'publisher) tend to use send-message and tend to +'publisher) tend to use send-message and tend to respond to feedback from subscribers; subscribers -('subscriber) tend to use send-feedback and respond -to messages from publishers.

type

InterestType : (U 'participant 'observer 'everything)

Using interest-type 'participant in an endpoint’s role +('subscriber) tend to use send-feedback and respond +to messages from publishers.

type

InterestType : (U 'participant 'observer 'everything)

Using interest-type 'participant in an endpoint’s role indicates that the endpoint is intending to act as a genuine participant in whatever protocol is associated with the endpoint and its topic.

Using 'observer indicates that the endpoint is intended to @@ -42,59 +44,63 @@ Endpoints with interest-type 'e relevant for managing demand for observers, as well as in some cases of cross-layer presence/absence propagation. Most programs (and even most drivers) will not need to use the 'everything -interest-type.

3.3 Endpoint Events

type

EndpointEvent : (U PresenceEvent AbsenceEvent MessageEvent)

type

PresenceEvent : presence-event

type

AbsenceEvent : absence-event

type

MessageEvent : message-event

Endpoint events are passed to handler functions by VMs, conveying some +interest-type.

3.3 Endpoint Events

Endpoint events are passed to handler functions by VMs, conveying some change in the world the process lives in. An endpoint event can signal the arrival or departure of a conversational peer, or can deliver a -message that has been sent on a VM’s IPC facility.

struct

(struct presence-event (role)
  #:prefab)
  role : Role
Indicates the arrival of a new conversational partner: an endpoint -with a topic that intersects our own, with Orientation -opposite to our own.

The presence-event-role describes the arriving peer, or more +message that has been sent on a VM’s IPC facility.

struct

(struct presence-event (role)
  #:prefab)
  role : Role
Indicates the arrival of a new conversational partner: an endpoint +with a topic that intersects our own, with Orientation +opposite to our own.

The presence-event-role describes the arriving peer, or more precisely, describes the shared interest between ourselves and the new -peer. In particular, the role-orientation of the -presence-event-role is the orientation that the peer -supplied in its add-endpoint structure.

struct

(struct absence-event (role reason)
  #:prefab)
  role : Role
  reason : Any
Indicates the departure of an existing conversational partner, through -either an explicit delete-endpoint action or the implicit -deleting of all of a process’s endpoints when a process exits.

The absence-event-role describes the departing peer, -analogously to presence-event-role.

struct

(struct message-event (role message)
  #:prefab)
  role : Role
  message : Message
Indicates the arrival of a message matching the topic pattern in the -handler’s endpoint.

3.4 Actions

type

Action : (All (State) (U (PreAction State) (yield State) (at-meta-level State)))

type

PreAction : (All (State) (U (add-endpoint State) delete-endpoint send-message (spawn State) quit))

Actions are requests from a process to its containing VM. If wrapped -in an at-meta-level structure, the action is to apply to +peer. In particular, the role-orientation of the +presence-event-role is the orientation that the peer +supplied in its add-endpoint structure.

struct

(struct absence-event (role reason)
  #:prefab)
  role : Role
  reason : Any
Indicates the departure of an existing conversational partner, through +either an explicit delete-endpoint action or the implicit +deleting of all of a process’s endpoints when a process exits.

The absence-event-role describes the departing peer, +analogously to presence-event-role.

struct

(struct message-event (role message)
  #:prefab)
  role : Role
  message : Message
Indicates the arrival of a message matching the topic pattern in the +handler’s endpoint.

3.4 Actions

type

Action : 
(All (State) (U (PreAction State)
                (yield State)
                (at-meta-level State)))

type

PreAction : 
(All (State) (U (add-endpoint State)
                delete-endpoint
                send-message
                (spawn State)
                quit))
Actions are requests from a process to its containing VM. If wrapped +in an at-meta-level structure, the action is to apply to the VM’s own containing VM; otherwise, the action applies to -the process’s containing VM.

struct

(struct at-meta-level (preaction)
  #:prefab)
  preaction : (PreAction State)

type

AtMetaLevel : (All (State) (at-meta-level State))

An at-meta-level structure wraps a plain action, and makes it -apply to the outer VM instead of the inner VM (the default).

struct

(struct yield (k)
  #:prefab)
  k : (InterruptK State)

type

Yield : (All (State) (yield State))

Because current VM implementations are cooperatively scheduled, it can +the process’s containing VM.

struct

(struct at-meta-level (preaction)
  #:prefab)
  preaction : (PreAction State)

type

AtMetaLevel : (All (State) (at-meta-level State))

An at-meta-level structure wraps a plain action, and makes it +apply to the outer VM instead of the inner VM (the default).

struct

(struct yield (k)
  #:prefab)
  k : (InterruptK State)

type

Yield : (All (State) (yield State))

Because current VM implementations are cooperatively scheduled, it can sometimes be necessary to explicitly yield the CPU to other processes -using a yield action. When control returns to the yielding -process, the yield-k is invoked.

3.4.1 Endpoints and Messages

struct

(struct add-endpoint (pre-eid role handler)
  #:prefab)
  pre-eid : Any
  role : Role
  handler : (Handler State)

type

AddEndpoint : (All (State) (add-endpoint State))

Creates a new endpoint subscribing to the given Role. When -events pertaining to the given role occur, the Handler is -invoked.If invoked at-meta-level, subscribes to events +using a yield action. When control returns to the yielding +process, the yield-k is invoked.

3.4.1 Endpoints and Messages

struct

(struct add-endpoint (pre-eid role handler)
  #:prefab)
  pre-eid : Any
  role : Role
  handler : (Handler State)

type

AddEndpoint : (All (State) (add-endpoint State))

Creates a new endpoint subscribing to the given Role. When +events pertaining to the given role occur, the Handler is +invoked.If invoked at-meta-level, subscribes to events in the containing VM’s container.

The name of the new endpoint will be the pre-eid; it must be unique within the current process, but otherwise can be any value at all. If the endpoint’s name matches an existing endpoint, and the new role is the same as the existing endpoint’s role, the handler function -is replaced in the existing endpoint.

To delete an endpoint, perform a delete-endpoint action built -with the name of the endpoint to delete.

struct

(struct delete-endpoint (pre-eid reason)
  #:prefab)
  pre-eid : Any
  reason : Any

type

DeleteEndpoint : delete-endpoint

Deletes an existing endpoint named pre-eid. The given +is replaced in the existing endpoint.

To delete an endpoint, perform a delete-endpoint action built +with the name of the endpoint to delete.

struct

(struct delete-endpoint (pre-eid reason)
  #:prefab)
  pre-eid : Any
  reason : Any

type

DeleteEndpoint : delete-endpoint

Deletes an existing endpoint named pre-eid. The given reason is passed along to peer endpoints as part of an -absence-event.

If no specific reason is needed, it is conventional to supply -#f as the delete-endpoint-reason.

struct

(struct send-message (body orientation)
  #:prefab)
  body : Message
  orientation : Orientation

type

SendMessage : send-message

Sends a message to peers.Or, if at-meta-level, peers of -the containing VM. The given Orientation should describe the +absence-event.

If no specific reason is needed, it is conventional to supply +#f as the delete-endpoint-reason. See also the +convenience delete-endpoint +function from marketplace/sugar-values.

struct

(struct send-message (body orientation)
  #:prefab)
  body : Message
  orientation : Orientation

type

SendMessage : send-message

Sends a message to peers.Or, if at-meta-level, peers of +the containing VM. The given Orientation should describe the role the sender is playing when sending this message: usually, it will be 'publisher, but when the message is feedback for -some publisher, it will be 'subscriber. See also -send-feedback.

3.4.2 Process Management

struct

(struct spawn (spec k debug-name)
  #:prefab)
  spec : process-spec
  k : (Option (PID -> (InterruptK State)))
  debug-name : Any

struct

(struct process-spec (boot)
  #:prefab)
  boot : (PID -> CoTransition)

type

CoTransition : 
(All (Result)
     (All (State) (Transition State) -> Result)
     -> Result)

type

Spawn : (All (State) (spawn State))

type

ProcessSpec : process-spec

A spawn requests the creation of a sibling processIf -wrapped in an at-meta-level, the new process will instead be -a sibling of the creating process’s VM.. The spawn-k runs in +some publisher, it will be 'subscriber. +See also the send-message and +send-feedback convenience functions from +marketplace/sugar-values.

3.4.2 Process Management

struct

(struct spawn (spec k debug-name)
  #:prefab)
  spec : process-spec
  k : (Option (PID -> (InterruptK State)))
  debug-name : Any

struct

(struct process-spec (boot)
  #:prefab)
  boot : (PID -> CoTransition)

type

CoTransition : 
(All (Result)
     (All (State) (Transition State) -> Result)
     -> Result)

type

Spawn : (All (State) (spawn State))

type

ProcessSpec : process-spec

A spawn requests the creation of a sibling processIf +wrapped in an at-meta-level, the new process will instead be +a sibling of the creating process’s VM.. The spawn-k runs in the context of the creating process, communicating to it the -PID of the new process.

The spawn-spec describes the new process to be created. Its -process-spec-boot field is a function taking the PID of the +PID of the new process.

The spawn-spec describes the new process to be created. Its +process-spec-boot field is a function taking the PID of the new process and returning a "cotransition". Cotransitions use a second-order encoding of existential types to guarantee that the VM remains oblivious to the specific process state type of the new process. The downside of this approach is its syntactic and type -complexity: see spawn: for an easier-to-use, higher-level -approach.

struct

(struct quit (pid reason)
  #:prefab)
  pid : (Option PID)
  reason : Any

type

Quit : quit

Kills a sibling process.Or, if at-meta-level, a sibling -process of the containing VM. If quit-pid is #f, +complexity: see spawn: for an easier-to-use, higher-level +approach.

struct

(struct quit (pid reason)
  #:prefab)
  pid : (Option PID)
  reason : Any

type

Quit : quit

Kills a sibling process.Or, if at-meta-level, a sibling +process of the containing VM. If quit-pid is #f, kills the current process; otherwise, kills the process with the given -PID. The quit-reason is passed on to peers of +PID. The quit-reason is passed on to peers of currently-active endpoints in the process to be killed, as part of a -absence-event, just as if each active endpoint were deleted +absence-event, just as if each active endpoint were deleted manually before the process exited.

If no specific reason is needed, it is conventional to supply -#f as the quit-reason.

type

PID : Number

In the current VM implementations, process IDs are simply numbers. +#f as the quit-reason.

type

PID : Number

In the current VM implementations, process IDs are simply numbers. PIDs are scoped to and allocated by each individual VM instance.

 
\ No newline at end of file diff --git a/pict.png b/pict.png index a87c41e..a0a84d7 100644 Binary files a/pict.png and b/pict.png differ diff --git a/pict_2.png b/pict_2.png index 3ccb1c2..f8e93ce 100644 Binary files a/pict_2.png and b/pict_2.png differ diff --git a/pict_3.png b/pict_3.png index 170ae9f..1bf20d0 100644 Binary files a/pict_3.png and b/pict_3.png differ