From 34271cf81c5606687a33152c83364d9fbf350c9f Mon Sep 17 00:00:00 2001 From: Tony Garnock-Jones Date: Mon, 20 May 2013 18:14:06 -0400 Subject: [PATCH] Update 20130520181406 --- Concepts.html | 4 ++-- Drivers.html | 23 +++++++++++++++++-- Examples.html | 4 ++-- Management_and_Monitoring.html | 8 +++---- high-level-interface.html | 41 +++++++++++++++++++++++++--------- index.html | 4 ++-- low-level-interface.html | 31 +++++++++++++++++++++++-- writing-new-drivers.html | 5 ----- 8 files changed, 90 insertions(+), 30 deletions(-) delete mode 100644 writing-new-drivers.html diff --git a/Concepts.html b/Concepts.html index a96c1ac..9deb4a9 100644 --- a/Concepts.html +++ b/Concepts.html @@ -1,5 +1,5 @@ -1 Concepts
▼Marketplace:   Network-Aware Programming
►1 Concepts
On this page:
1.1 What is a process, what are event handlers?
1.2 What is a VM?
1.3 Endpoints:   Subscription and Advertisement
1.4 Messages and Topics
1.5 Presence
1.6 Nesting, relaying, and levels of discourse
5.3.4.10
    ← prev  up  next → 

1 Concepts

Marketplace integrates ideas from both distributed systems and +1 Concepts

▼Marketplace:   Network-Aware Programming
►1 Concepts
On this page:
1.1 What is a process, what are event handlers?
1.2 What is a VM?
1.3 Endpoints:   Subscription and Advertisement
1.4 Messages and Topics
1.5 Presence
1.6 Nesting, relaying, and levels of discourse
5.3.4.10
    ← prev  up  next → 

1 Concepts

Marketplace integrates ideas from both distributed systems and virtualized operating system designs to obtain an architecture of nested virtual machines (VMs). Each nested layer is equipped with its own publish/subscribe network that also propagates @@ -15,7 +15,7 @@ carries a VM-specific protocol that is often different from the protocol spoken by its containing VM.

The outermost VM is called the ground VM. The protocol spoken by processes running within the ground VM is a simple protocol relating Racket’s synchronizable events to Marketplace network -messages. See Writing New Drivers and Drivers for +messages. See Drivers for information on using Racket events from Marketplace programs.

1.1 What is a process, what are event handlers?

A Marketplace process is a collection of event handlers, plus a piece of private process state. Every processThe exception to this rule is the Ground VM, which plays diff --git a/Drivers.html b/Drivers.html index 9e05d53..10c67a1 100644 --- a/Drivers.html +++ b/Drivers.html @@ -1,5 +1,5 @@ -4 Drivers

▼Marketplace:   Network-Aware Programming
►4 Drivers
On this page:
4.1 event-relay
event-relay
4.2 tcp-bare
tcp-driver
4.2.1 TCP channels
tcp-channel
Tcp  Sub  Packet
4.2.2 TCP addresses
Tcp  Address
tcp-address
tcp-handle
tcp-listener
4.2.3 Opening an outbound connection
4.2.4 Accepting inbound connections
4.2.5 Receiving data
4.2.6 Sending data
4.3 tcp
4.4 timer (typed and untyped)
4.5 udp (typed and untyped)
5.3.4.10
    ← prev  up  next → 

4 Drivers

4.1 event-relay

 (require marketplace/drivers/event-relay)

procedure

(event-relay self-id) → Spawn

  self-id : Symbol
Lets processes in some nested-vm interact with the outside +4 Drivers
▼Marketplace:   Network-Aware Programming
►4 Drivers
On this page:
4.1 event-relay
event-relay
4.2 tcp-bare
tcp-driver
4.2.1 TCP channels
tcp-channel
Tcp  Sub  Packet
4.2.2 TCP addresses
Tcp  Address
tcp-address
tcp-handle
tcp-listener
4.2.3 Opening an outbound connection
4.2.4 Accepting inbound connections
4.2.5 Receiving data
4.2.6 Sending data
4.3 timer (typed and untyped)
4.4 udp (typed and untyped)
5.3.4.10
    ← prev  up  next → 

4 Drivers

4.1 event-relay

 (require marketplace/drivers/event-relay)

procedure

(event-relay self-id) → Spawn

  self-id : Symbol
Lets processes in some nested-vm interact with the outside world using ground-vm-level event-based subscriptions.

Returns a spawn which starts an event-relay process with debug-name `(event-relay ,self-id).
The relay process observes subscriptions matching the topic-pattern (cons (? evt?) _), and when one appears, constructs an @@ -27,4 +27,23 @@ sure not to accidentally clash in handle ID selection. They are also used in TcpChannel to mean a specific instance of a TCP connection, so if you are likely to want to reconnect individual flows, use different values for id.

struct

(struct tcp-listener (port)
  #:prefab)
  port : (integer-in 0 65535)
Describes a local half-connection with a user-assigned port number. -Use this to describe server sockets.

4.2.3 Opening an outbound connection

TODO

4.2.4 Accepting inbound connections

TODO

4.2.5 Receiving data

TODO

4.2.6 Sending data

TODO

4.3 tcp

TODO

4.4 timer (typed and untyped)

TODO

4.5 udp (typed and untyped)

TODO

    ← prev  up  next → 
 
\ No newline at end of file +Use this to describe server sockets.

4.2.3 Opening an outbound connection

Choose a tcp-handle, and then create endpoints as follows:

(let ((local (tcp-handle 'some-unique-value))
      (remote (tcp-address "the.remote.host.example.com" 5999)))
  (transition/no-state
   (endpoint #:publisher (tcp-channel local remote ?))
   (endpoint #:subscriber (tcp-channel remote local ?)
             [(tcp-channel _ _ (? eof-object?))
              ; Handle a received end-of-file object
              (transition ...)]
             [(tcp-channel _ _ (? bytes? data))
              ; Handle received data
              (transition ...)])))

The TCP driver will automatically create an outbound connection in +response to the presence of the endpoints. When the endpoints are +deleted (or the process exits), the TCP driver will notice the absence +and will close the underlying TCP socket.

For a complete example, see TCP chat client.

4.2.4 Accepting inbound connections

Choose a port number, and then create an observer endpoint as +follows:

(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) #:observer
          #:conversation (tcp-channel them us _)
          #:on-presence (spawn #:child (chat-session them us)))

The use of #:observer here indicates that this endpoint isn’t +actually interested in exchanging any TCP data; instead, it is +monitoring demand for such exchanges. The TCP driver uses a rare +#:everything endpoint to monitor the presence of +#:observers, and creates listening TCP server sockets in +response. When a connection comes in, the TCP driver spawns a manager +process which offers regular #:participant endpoints for +communicating on the newly-arrived socket.

To illustrate the code for handling a newly-arrived connection,

(define (chat-session them us)
  (transition/no-state
   (endpoint #:subscriber (tcp-channel them us ?)
             #:on-absence (quit)
             [(tcp-channel _ _ (? bytes? data))
              ; Handle incoming data
              (transition ...)])))
4.2.5 Receiving data

TCP-related messages will be of the form

(tcp-channel remote-address local-address subpacket)

where the subpacket is either eof or a +bytes?.

4.2.6 Sending data

Send data with

(send-message (tcp-channel local-address remote-address subpacket))

where, as for receiving data, the subpacket is either +eof or a bytes?.

4.3 timer (typed and untyped)

For examples of the use of the timer driver, see uses of +set-timer and timer-expired in +the +Marketplace-based DNS resolver.

4.4 udp (typed and untyped)

For examples of the use of the UDP driver, see uses of +udp-packet etc. in +the +Marketplace-based DNS resolver.

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/Examples.html b/Examples.html index 22ea26d..2a7c75a 100644 --- a/Examples.html +++ b/Examples.html @@ -1,5 +1,5 @@ -7 Examples
▼Marketplace:   Network-Aware Programming
►7 Examples
On this page:
7.1 TCP echo server
7.2 TCP chat server
7.3 TCP chat client
5.3.4.10
    ← prev  up  next → 

7 Examples

7.1 TCP echo server

Here is a complete Marketplace program:

"examples/echo-paper.rkt"

#lang marketplace
 
(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?)
          #:conversation (tcp-channel from to _)
          #:on-presence (spawn #:child (echoer from to)))
 
(define (echoer from to)
(transition stateless
(endpoint #:subscriber (tcp-channel from to ?)
          #:on-absence (quit)
          [(tcp-channel _ _ data)
           (send-message (tcp-channel to from data))])))

The top-level endpoint action subscribes to TCP connections +6 Examples

▼Marketplace:   Network-Aware Programming
►6 Examples
On this page:
6.1 TCP echo server
6.2 TCP chat server
6.3 TCP chat client
5.3.4.10
    ← prev  up  next → 

6 Examples

6.1 TCP echo server

Here is a complete Marketplace program:

"examples/echo-paper.rkt"

#lang marketplace
 
(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?)
          #:conversation (tcp-channel from to _)
          #:on-presence (spawn #:child (echoer from to)))
 
(define (echoer from to)
(transition stateless
(endpoint #:subscriber (tcp-channel from to ?)
          #:on-absence (quit)
          [(tcp-channel _ _ data)
           (send-message (tcp-channel to from data))])))

The top-level endpoint action subscribes to TCP connections arriving on port 5999, and spawns a fresh process in response to each (#:on-presence). The topic of conversation (#:conversation) associated with the newly-present @@ -16,4 +16,4 @@ the #:on-absence handler in echo action, terminating the connection’s process. The heart of our system is the interface between a process and its containing VM. Our implementation instantiates this interface as a collection of Typed -Racket programs.

7.2 TCP chat server

"examples/chat-paper.rkt"

#lang marketplace
 
(nested-vm
 (at-meta-level
  (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) #:observer
            #:conversation (tcp-channel them us _)
            #:on-presence (spawn #:child (chat-session them us)))))
 
(define (chat-session them us)
(define user (gensym 'user))
(transition stateless
            (listen-to-user user them us)
            (speak-to-user user them us)))
 
(define (listen-to-user user them us)
(list
 (endpoint #:publisher `(,user says ,?))
 (at-meta-level
  (endpoint #:subscriber (tcp-channel them us ?)
            #:on-absence (quit)
            [(tcp-channel _ _ (? bytes? text))
             (send-message `(,user says ,text))]))))
 
(define (speak-to-user user them us)
(define (say fmt . args)
(at-meta-level (send-message (tcp-channel us them (apply format fmt args)))))
(define (announce who did-what)
(unless (equal? who user) (say "~s ~s.~n" who did-what)))
(list
 (say "You are ~s.~n" user)
 (at-meta-level
  (endpoint #:publisher (tcp-channel us them ?)))
(endpoint #:subscriber `(,? says ,?)
          #:conversation `(,who says ,_)
          #:on-presence (announce who 'arrived)
          #:on-absence  (announce who 'departed)
          [`(,who says ,what) (say "~a: ~a" who what)])))

7.3 TCP chat client

"examples/chat-client.rkt"

#lang marketplace
(require racket/port)
 
(spawn #:debug-name 'console-output-driver
       #:child
       (transition/no-state
        (endpoint #:subscriber (list 'console-output ?)
                  [(list 'console-output item)
                   (begin (printf "~a" item)
                          (void))])))
 
(spawn #:debug-name 'console-input-driver
       #:child
       (transition/no-state
        (endpoint #:publisher (list 'console-input ?)
                  #:name 'input-relay
                  #:on-absence
                  (list (send-message (list 'console-output "Connection terminated.\n"))
                        (quit)))
(endpoint #:subscriber (cons (read-line-evt (current-input-port) 'any) ?)
          [(cons _ (? eof-object?))
           (list (send-message (list 'console-output "Terminating on local EOF.\n"))
                 (delete-endpoint 'input-relay))]
[(cons _ (? string? line))
 (send-message (list 'console-input line))])))
 
(spawn #:debug-name 'outbound-connection
       #:child
       (let ((local (tcp-handle 'outbound))
             (remote (tcp-address "localhost" 5999)))
(transition/no-state
 (endpoint #:subscriber (list 'console-input ?)
           #:on-absence (quit)
           [(list 'console-input line)
            (list (send-message (list 'console-output (format "> ~a \n" line)))
                  (send-message (tcp-channel local remote (string-append line "\n"))))])
(endpoint #:publisher (tcp-channel local remote ?))
(endpoint #:subscriber (tcp-channel remote local ?)
          #:on-absence (quit)
          [(tcp-channel _ _ (? eof-object?))
           (quit)]
[(tcp-channel _ _ data)
 (list (send-message (list 'console-output (format "< ~a" data)))
       (void))]))))
    ← prev  up  next → 
 
\ No newline at end of file +Racket programs.

6.2 TCP chat server

"examples/chat-paper.rkt"

#lang marketplace
 
(nested-vm
 (at-meta-level
  (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) #:observer
            #:conversation (tcp-channel them us _)
            #:on-presence (spawn #:child (chat-session them us)))))
 
(define (chat-session them us)
(define user (gensym 'user))
(transition stateless
            (listen-to-user user them us)
            (speak-to-user user them us)))
 
(define (listen-to-user user them us)
(list
 (endpoint #:publisher `(,user says ,?))
 (at-meta-level
  (endpoint #:subscriber (tcp-channel them us ?)
            #:on-absence (quit)
            [(tcp-channel _ _ (? bytes? text))
             (send-message `(,user says ,text))]))))
 
(define (speak-to-user user them us)
(define (say fmt . args)
(at-meta-level (send-message (tcp-channel us them (apply format fmt args)))))
(define (announce who did-what)
(unless (equal? who user) (say "~s ~s.~n" who did-what)))
(list
 (say "You are ~s.~n" user)
 (at-meta-level
  (endpoint #:publisher (tcp-channel us them ?)))
(endpoint #:subscriber `(,? says ,?)
          #:conversation `(,who says ,_)
          #:on-presence (announce who 'arrived)
          #:on-absence  (announce who 'departed)
          [`(,who says ,what) (say "~a: ~a" who what)])))

6.3 TCP chat client

"examples/chat-client.rkt"

#lang marketplace
(require racket/port)
 
(spawn #:debug-name 'console-output-driver
       #:child
       (transition/no-state
        (endpoint #:subscriber (list 'console-output ?)
                  [(list 'console-output item)
                   (begin (printf "~a" item)
                          (void))])))
 
(spawn #:debug-name 'console-input-driver
       #:child
       (transition/no-state
        (endpoint #:publisher (list 'console-input ?)
                  #:name 'input-relay
                  #:on-absence
                  (list (send-message (list 'console-output "Connection terminated.\n"))
                        (quit)))
(endpoint #:subscriber (cons (read-line-evt (current-input-port) 'any) ?)
          [(cons _ (? eof-object?))
           (list (send-message (list 'console-output "Terminating on local EOF.\n"))
                 (delete-endpoint 'input-relay))]
[(cons _ (? string? line))
 (send-message (list 'console-input line))])))
 
(spawn #:debug-name 'outbound-connection
       #:child
       (let ((local (tcp-handle 'outbound))
             (remote (tcp-address "localhost" 5999)))
(transition/no-state
 (endpoint #:subscriber (list 'console-input ?)
           #:on-absence (quit)
           [(list 'console-input line)
            (list (send-message (list 'console-output (format "> ~a \n" line)))
                  (send-message (tcp-channel local remote (string-append line "\n"))))])
(endpoint #:publisher (tcp-channel local remote ?))
(endpoint #:subscriber (tcp-channel remote local ?)
          #:on-absence (quit)
          [(tcp-channel _ _ (? eof-object?))
           (quit)]
[(tcp-channel _ _ data)
 (list (send-message (list 'console-output (format "< ~a" data)))
       (void))]))))
    ← prev  up  next → 
 
\ No newline at end of file diff --git a/Management_and_Monitoring.html b/Management_and_Monitoring.html index 8e2a0b9..a13422f 100644 --- a/Management_and_Monitoring.html +++ b/Management_and_Monitoring.html @@ -1,11 +1,11 @@ -6 Management and Monitoring
▼Marketplace:   Network-Aware Programming
►6 Management and Monitoring
On this page:
6.1 generic-spy
generic-spy
6.2 logging (MATRIX_  LOG)
MATRIX_  LOG
matrix-log
matrix-root-logger
6.3 debugger (experimental)
debug
5.3.4.10
    ← prev  up  next → 

6 Management and Monitoring

6.1 generic-spy

 (require marketplace/support/spy)

procedure

(generic-spy label) → Spawn

  label : Any
Returns a spawn action that, when executed, creates a process +5 Management and Monitoring
▼Marketplace:   Network-Aware Programming
►5 Management and Monitoring
On this page:
5.1 generic-spy
generic-spy
5.2 logging (MATRIX_  LOG)
MATRIX_  LOG
matrix-log
matrix-root-logger
5.3 debugger (experimental)
debug
5.3.4.10
    ← prev  up  next → 

5 Management and Monitoring

5.1 generic-spy

 (require marketplace/support/spy)

procedure

(generic-spy label) → Spawn

  label : Any
Returns a spawn action that, when executed, creates a process with a #:subscriber endpoint listening for every message. Each EndpointEvent received by the endpoint is printed to the current output port. Using this process gives a crude trace of activity within a VM: presence-events and absence-events (of #:publishers) are logged, as is -each 'publisher message sent to the VM’s network.

6.2 logging (MATRIX_LOG)

 (require marketplace/log-untyped)
 (require marketplace/log-typed)

environment variable

MATRIX_LOG

Set the MATRIX_LOG environment variable to "debug", "info", +each 'publisher message sent to the VM’s network.

5.2 logging (MATRIX_LOG)

 (require marketplace/log-untyped)
 (require marketplace/log-typed)

environment variable

MATRIX_LOG

Set the MATRIX_LOG environment variable to "debug", "info", "warning", "error" or "fatal" (i.e. any of Racket’s log-level?s) to enable output of log messages at that level and higher.

If MATRIX_LOG is not defined in the environment, no log @@ -13,9 +13,9 @@ output will be produced.

matrix-root-logger instead of the system logger. The level expression must evaluate to a level symbol (see log-level?), and format-str must evaluate to a -format string for use with format.

value

matrix-root-logger : logger?

The root logger for marketplace logging.

6.3 debugger (experimental)

 (require marketplace/support/debug)

procedure

(debug p) → Spawn

  p : Spawn
Translates a spawn action to another spawn action which wraps +format string for use with format.

value

matrix-root-logger : logger?

The root logger for marketplace logging.

5.3 debugger (experimental)

 (require marketplace/support/debug)

procedure

(debug p) → Spawn

  p : Spawn
Translates a spawn action to another spawn action which wraps the to-be-spawned process in a debugging interface. Executing the resulting action will not only create a process in the executing VM, but will also open a debugger GUI.

N.B.: The debugger is experimental and likely to change quite quickly and unpredictably.
See the file "examples/debug-chat.rkt" for an example of the -use of debug.

    ← prev  up  next → 
 
\ No newline at end of file +use of debug.

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/high-level-interface.html b/high-level-interface.html index 0f18d4f..e645848 100644 --- a/high-level-interface.html +++ b/high-level-interface.html @@ -1,19 +1,19 @@ -2 High-level interface
▼Marketplace:   Network-Aware Programming
►2 High-level interface
On this page:
2.1 Using #lang marketplace and friends
2.2 Using Marketplace as a library
ground-vm
ground-vm:
2.3 Constructing transitions
2.4 Creating endpoints
2.4.1 Receiving messages
2.4.2 Action-only vs. State updates
2.4.3 Naming endpoints
2.4.4 Handling presence and absence events
2.4.5 Exit reasons
2.4.6 Updating endpoints
2.4.7 Who am I talking to?
2.5 Deleting endpoints
2.6 Sending messages and feedback
2.7 Creating processes
2.8 Exiting and killing processes
2.9 Cooperative scheduling
2.10 Creating nested VMs
2.11 Relaying across layers
5.3.4.10
    ← prev  up  next → 

2 High-level interface

This high-level interface between a VM and a process is analogous to +2 High-level interface

▼Marketplace:   Network-Aware Programming
►2 High-level interface
On this page:
2.1 Using #lang marketplace and friends
2.2 Using Marketplace as a library
ground-vm
ground-vm:
2.3 Constructing transitions
2.4 Creating endpoints
2.4.1 Receiving messages
2.4.2 Action-only vs. State updates
2.4.3 Naming endpoints
2.4.4 Handling presence and absence events
2.4.5 Exit reasons
2.4.6 Updating endpoints
2.4.7 Who am I talking to?
2.5 Deleting endpoints
2.6 Sending messages and feedback
2.7 Creating processes
2.8 Exiting and killing processes
2.9 Cooperative scheduling
2.10 Creating nested VMs
2.11 Relaying across layers
5.3.4.10
    ← prev  up  next → 

2 High-level interface

This high-level interface between a VM and a process is analogous to the C library interface of a Unix-like operating system. The Low-level interface corresponds to the system call -interface of a Unix-like operating system.

2.1 Using #lang marketplace and friends

 #lang marketplace
 #lang marketplace/flow-control
 #lang marketplace/typed
 #lang marketplace/typed/flow-control

Programs written for Marketplace differ from normal Racket modules +interface of a Unix-like operating system.

2.1 Using #lang marketplace and friends

 #lang marketplace

Programs written for Marketplace differ from normal Racket modules only in their selection of language. A Racket module written with #lang marketplace, such as the echo server in TCP echo server, specifies a sequence of definitions and startup actions for an application. Typically, initial actions spawn application processes and nested VMs, which in turn -subscribe to sources of events from the outside world.

There are a handful of variant languages to choose from:

  • marketplace is for untyped programs, and uses -the tcp-bare TCP driver;

  • marketplace/flow-control is like -marketplace, but uses the flow-controlled tcp -driver;

  • marketplace/typed is like marketplace, but -for typed programs;

  • marketplace/typed/flow-control is like -marketplace/flow-control, but for typed programs.

2.2 Using Marketplace as a library

 (require marketplace/sugar-untyped)
 (require marketplace/sugar-typed)

Instead of using Racket’s #lang feature, ordinary Racket programs +subscribe to sources of events from the outside world.

At present, there’s just #lang marketplace. In future, there will +be a variation for Typed Racket, and languages providing greater +support for flow control, responsibility transfer, and other +networking concepts. For now, Typed Racket programs must be written as +#lang typed/racket programs using (require marketplace) +and ground-vm: explicitly.

2.2 Using Marketplace as a library

 (require marketplace/sugar-untyped)
 (require marketplace/sugar-typed)

Instead of using Racket’s #lang feature, ordinary Racket programs can use Marketplace features by requiring Marketplace modules directly.

Such programs need to use ground-vm/ground-vm: to start the ground-level VM explicitly. They also need to explicitly @@ -147,6 +147,25 @@ action. In that case, the reason will be the raised itself.

2.9 Cooperative scheduling

syntax

(yield maybe-state-pattern k-expr)

syntax

(yield: typed-state-pattern k-expr)

 
maybe-state-pattern = 
  | #:state pattern
     
typed-state-pattern = : State
  | pattern : State
     
k-expr = expr
Lets other processes in the system run for a step, returning to evaluate k-expr only after doing a complete round of the scheduler.

If pattern is supplied, k-expr should evaluate to a -Transition; otherwise it should produce an ActionTree.

2.10 Creating nested VMs

***** nested-vm, nested-vm: -TODO

2.11 Relaying across layers

***** at-meta-level, at-meta-level: -TODO

    ← prev  up  next → 
 
\ No newline at end of file +Transition; otherwise it should produce an ActionTree.

2.10 Creating nested VMs

syntax

(nested-vm maybe-vm-pid-binding maybe-boot-pid-binding
           maybe-initial-state
           maybe-debug-name
           boot-action-expr ...)

syntax

(nested-vm: : ParentStateType
            maybe-vm-pid-binding maybe-boot-pid-binding
            maybe-typed-initial-state
            maybe-debug-name
            boot-action-expr ...)
 
maybe-vm-pid-binding = 
  | #:vm-pid identifier
     
maybe-boot-pid-binding = 
  | #:boot-pid identifier
     
maybe-initial-state = 
  | #:initial-state expr
     
maybe-typed-initial-state = 
  | #:initial-state expr : StateType
     
maybe-debug-name = 
  | #:debug-name expr
     
boot-action-expr = expr
Results in a spawn action that starts a nested VM. The +primordial process in the new VM executes the boot-actions with the +given initial state. (If no initial state is supplied, (void) +is used.)

If #:vm-pid is present, the corresponding identifier is bound +in the boot-action expressions to the container-relative PID of the +new VM itself. If #:boot-pid is present, however, the +corresponding identifier is bound to the new-VM-relative PID of the +primordial process in the new VM.

2.11 Relaying across layers

syntax

(at-meta-level: : StateType preaction ...)

procedure

(at-meta-level preaction ...) → (Action StateType)

  preaction : (PreAction State)
Each VM gives its processes access to two distinct IPC facilities: the +internal one, provided for the VM’s processes to talk amongst +themselves, and the external one, the network that the VM +itself is a process within.

Marketplace’s actions can apply to either of those two networks. By +default, actions apply to the VM of the acting process directly, but +using at-meta-level (or at-meta-level: in typed +code) to wrap an action level-shifts the action to make it +apply at the level of the acting process’s VM’s container instead.

For example, wrapping an endpoint in at-meta-level +adds a subscription to the VM’s container’s network. Instead of +listening to sibling processes of the acting process, the new endpoint +will listen to sibling processes of the acting process’s VM. In this +example, the primordial process in the nested-vm creates an +endpoint in the VM’s own network, the ground VM:

(nested-vm
 (at-meta-level
  (endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) ...)))

In this example, a new process is spawned as a sibling of the +nested-vm rather than as a sibling of its primordial process:

(nested-vm
 (at-meta-level
  (spawn #:child (transition/no-state (send-message 'hello-world)))))

Compare to this example, which spawns a sibling of the +nested-vm’s primordial process:

(nested-vm
 (spawn #:child (transition/no-state (send-message 'hello-world))))
    ← prev  up  next → 
 
\ No newline at end of file diff --git a/index.html b/index.html index 43eb972..f5fee15 100644 --- a/index.html +++ b/index.html @@ -1,5 +1,5 @@ -Marketplace: Network-Aware Programming
▼Marketplace:   Network-Aware Programming
5.3.4.10
    ← prev  up  next → 

Marketplace: Network-Aware Programming

Tony Garnock-Jones <tonyg@ccs.neu.edu>

Every program is a network. +Marketplace: Network-Aware Programming

▼Marketplace:   Network-Aware Programming
5.3.4.10
    ← prev  up  next → 

Marketplace: Network-Aware Programming

Tony Garnock-Jones <tonyg@ccs.neu.edu>

Every program is a network. This is the insight behind the π-calculus. Encoding a program as a π-calculus term shows it as a network of communicating processes. It is also one of the original inspirations for Smalltalk, where every @@ -34,4 +34,4 @@ location of mutable state, failure detection and recovery, access control, I/O and user interface, debugging and profiling.

Marketplace addresses these concerns with a small set of primitives chosen to make network programming in-the-small as flexible, scalable, manageable and securable as network programming in-the-large—and -vice versa.

    1 Concepts

      1.1 What is a process, what are event handlers?

      1.2 What is a VM?

      1.3 Endpoints: Subscription and Advertisement

      1.4 Messages and Topics

      1.5 Presence

      1.6 Nesting, relaying, and levels of discourse

    2 High-level interface

      2.1 Using #lang marketplace and friends

      2.2 Using Marketplace as a library

      2.3 Constructing transitions

      2.4 Creating endpoints

        2.4.1 Receiving messages

        2.4.2 Action-only vs. State updates

        2.4.3 Naming endpoints

        2.4.4 Handling presence and absence events

        2.4.5 Exit reasons

        2.4.6 Updating endpoints

        2.4.7 Who am I talking to?

      2.5 Deleting endpoints

      2.6 Sending messages and feedback

      2.7 Creating processes

      2.8 Exiting and killing processes

      2.9 Cooperative scheduling

      2.10 Creating nested VMs

      2.11 Relaying across layers

    3 Low-level interface

      3.1 Handler Functions

      3.2 Topics and Roles

      3.3 Endpoint Events

      3.4 Actions

        3.4.1 Endpoints and Messages

        3.4.2 Process Management

    4 Drivers

      4.1 event-relay

      4.2 tcp-bare

        4.2.1 TCP channels

        4.2.2 TCP addresses

        4.2.3 Opening an outbound connection

        4.2.4 Accepting inbound connections

        4.2.5 Receiving data

        4.2.6 Sending data

      4.3 tcp

      4.4 timer (typed and untyped)

      4.5 udp (typed and untyped)

    5 Writing New Drivers

    6 Management and Monitoring

      6.1 generic-spy

      6.2 logging (MATRIX_LOG)

      6.3 debugger (experimental)

    7 Examples

      7.1 TCP echo server

      7.2 TCP chat server

      7.3 TCP chat client

    ← prev  up  next → 
 
\ No newline at end of file +vice versa.

    1 Concepts

      1.1 What is a process, what are event handlers?

      1.2 What is a VM?

      1.3 Endpoints: Subscription and Advertisement

      1.4 Messages and Topics

      1.5 Presence

      1.6 Nesting, relaying, and levels of discourse

    2 High-level interface

      2.1 Using #lang marketplace and friends

      2.2 Using Marketplace as a library

      2.3 Constructing transitions

      2.4 Creating endpoints

        2.4.1 Receiving messages

        2.4.2 Action-only vs. State updates

        2.4.3 Naming endpoints

        2.4.4 Handling presence and absence events

        2.4.5 Exit reasons

        2.4.6 Updating endpoints

        2.4.7 Who am I talking to?

      2.5 Deleting endpoints

      2.6 Sending messages and feedback

      2.7 Creating processes

      2.8 Exiting and killing processes

      2.9 Cooperative scheduling

      2.10 Creating nested VMs

      2.11 Relaying across layers

    3 Low-level interface

      3.1 Handler Functions

      3.2 Topics and Roles

      3.3 Endpoint Events

      3.4 Actions

        3.4.1 Endpoints and Messages

        3.4.2 Process Management

    4 Drivers

      4.1 event-relay

      4.2 tcp-bare

        4.2.1 TCP channels

        4.2.2 TCP addresses

        4.2.3 Opening an outbound connection

        4.2.4 Accepting inbound connections

        4.2.5 Receiving data

        4.2.6 Sending data

      4.3 timer (typed and untyped)

      4.4 udp (typed and untyped)

    5 Management and Monitoring

      5.1 generic-spy

      5.2 logging (MATRIX_LOG)

      5.3 debugger (experimental)

    6 Examples

      6.1 TCP echo server

      6.2 TCP chat server

      6.3 TCP chat client

    ← prev  up  next → 
 
\ No newline at end of file diff --git a/low-level-interface.html b/low-level-interface.html index 7ba5dba..de1e9b0 100644 --- a/low-level-interface.html +++ b/low-level-interface.html @@ -1,5 +1,5 @@ -3 Low-level interface
▼Marketplace:   Network-Aware Programming
►3 Low-level interface
On this page:
3.1 Handler Functions
Handler
Trap  K
Interrupt  K
3.2 Topics and Roles
Topic
?
role
Role
Orientation
Interest  Type
3.3 Endpoint Events
Endpoint  Event
Presence  Event
Absence  Event
Message  Event
presence-event
absence-event
message-event
3.4 Actions
Action
Pre  Action
at-meta-level
At  Meta  Level
yield
Yield
3.4.1 Endpoints and Messages
add-endpoint
Add  Endpoint
delete-endpoint
Delete  Endpoint
send-message
Send  Message
3.4.2 Process Management
spawn
process-spec
Co  Transition
Spawn
Process  Spec
quit
Quit
PID
5.3.4.10
    ← prev  up  next → 

3 Low-level interface

 (require marketplace)

At its heart, the interface between each process and its +3 Low-level interface

▼Marketplace:   Network-Aware Programming
►3 Low-level interface
On this page:
3.1 Handler Functions
Handler
Trap  K
Interrupt  K
3.2 Topics and Roles
Topic
?
role
Role
Orientation
Interest  Type
3.3 Endpoint Events
Endpoint  Event
Presence  Event
Absence  Event
Message  Event
presence-event
absence-event
message-event
3.4 Actions
Action
Pre  Action
at-meta-level
At  Meta  Level
yield
Yield
3.4.1 Endpoints and Messages
add-endpoint
Add  Endpoint
delete-endpoint
Delete  Endpoint
send-message
Send  Message
3.4.2 Process Management
spawn
process-spec
Co  Transition
Spawn
Process  Spec
quit
Quit
PID
5.3.4.10
    ← prev  up  next → 

3 Low-level interface

 (require marketplace)

At its heart, the interface between each process and its containing VM is based on handler functions exchanging event and action structures with the VM. Both events and actions are simple Racket structures.

This low-level interface between a VM and a process is analogous to @@ -15,7 +15,34 @@ which may have embedded wildcards.

struct

(struct role (orientation topic interest-type)
  #:prefab)
  orientation : Orientation
  topic : Topic
  interest-type : InterestType

type

Role : role

Roles are almost always constructed by the endpoint/endpoint: macros or by the VM implementations themselves. User programs generally only need to -destructure role instances.

TODO: Role

type

Orientation : (U 'publisher 'subscriber)

TODO: Orientation

type

InterestType : (U 'participant 'observer 'everything)

TODO: InterestType

3.3 Endpoint Events

type

EndpointEvent : (U PresenceEvent AbsenceEvent MessageEvent)

type

PresenceEvent : presence-event

type

AbsenceEvent : absence-event

type

MessageEvent : message-event

Endpoint events are passed to handler functions by VMs, conveying some +destructure role instances.

A role describes the conversational role of a peer as seen by +some process. For example, a subscriber to topic 'foo with +interest-type 'participant might receive a presence +notification carrying the role

(role 'publisher 'foo 'participant)

Notice that the orientation of the role is the opposite of the +orientation of the endpoint.

type

Orientation : (U 'publisher 'subscriber)

Describes an endpoint’s orientation: will it be acting as a publisher +of messages, or as a subscriber to messages? Publishers (orientation +'publisher) tend to use send-message and tend to +respond to feedback from subscribers; subscribers +('subscriber) tend to use send-feedback and respond +to messages from publishers.

type

InterestType : (U 'participant 'observer 'everything)

Using interest-type 'participant in an endpoint’s role +indicates that the endpoint is intending to act as a genuine +participant in whatever protocol is associated with the endpoint and +its topic.

Using 'observer indicates that the endpoint is intended to +monitor other ongoing (participant) conversations instead. +Observer endpoints receive presence and absence notifications about +participant endpoints, but participant endpoints only receive +notifications about other participant endpoints, and not about +observer endpoints.

The 'observer interest-type is intended to make it easier to +monitor resource demand and supply. The monitoring endpoints/processes +can react to changing demand by creating or destroying resources to +match.

Finally, the 'everything interest-type receives notifications +about presence and absence of all the types of endpoint, +'participant, 'observer, and 'everything. +Endpoints with interest-type 'everything are rare: they are +relevant for managing demand for observers, as well as in some +cases of cross-layer presence/absence propagation. Most programs (and +even most drivers) will not need to use the 'everything +interest-type.

3.3 Endpoint Events

type

EndpointEvent : (U PresenceEvent AbsenceEvent MessageEvent)

type

PresenceEvent : presence-event

type

AbsenceEvent : absence-event

type

MessageEvent : message-event

Endpoint events are passed to handler functions by VMs, conveying some change in the world the process lives in. An endpoint event can signal the arrival or departure of a conversational peer, or can deliver a message that has been sent on a VM’s IPC facility.

struct

(struct presence-event (role)
  #:prefab)
  role : Role
Indicates the arrival of a new conversational partner: an endpoint diff --git a/writing-new-drivers.html b/writing-new-drivers.html deleted file mode 100644 index 4468321..0000000 --- a/writing-new-drivers.html +++ /dev/null @@ -1,5 +0,0 @@ - -5 Writing New Drivers
▼Marketplace:   Network-Aware Programming
5.3.4.10
    ← prev  up  next → 

5 Writing New Drivers

*** ground-vm’s usage of Racket evt?s -TODO -example of chat-client.rkt

*** pseudo-substruct -TODO

    ← prev  up  next → 
 
\ No newline at end of file