2 High-level interface

This high-level interface between a VM and a process is analogous to the C library interface of a Unix-like operating system. The Low-level interface corresponds to the system call interface of a Unix-like operating system.

2.1 Using #lang marketplace and friends

Programs written for Marketplace differ from normal Racket modules only in their selection of language. A Racket module written with #lang marketplace, such as the echo server in TCP echo server, specifies a sequence of definitions and startup actions for an application. Typically, initial actions spawn application processes and nested VMs, which in turn subscribe to sources of events from the outside world.

At present, there’s just #lang marketplace. In future, there will be a variation for Typed Racket, and languages providing greater support for flow control, responsibility transfer, and other networking concepts. For now, Typed Racket programs must be written as #lang typed/racket programs using (require marketplace) and ground-vm: explicitly.

2.2 Using Marketplace as a library

Instead of using Racket’s #lang feature, ordinary Racket programs can use Marketplace features by requiring Marketplace modules directly.

Such programs need to use ground-vm/ground-vm: to start the ground-level VM explicitly. They also need to explicitly start any drivers they need; for example, the file "examples/echo-plain.rkt" uses ground-vm along with tcp and an initial endpoint action:

(ground-vm

tcp

(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?)

#:conversation (tcp-channel from to _)

#:on-presence (spawn #:child (echoer from to))))

2.3 Constructing transitions

Each action-tree must be an (ActionTree State).

It’s fine to include no action-trees, in which case the transition merely updates the state of the process without taking any actions.

In the case of transition/no-state, the type Void and value (void) is used for the process state. transition/no-state is useful for processes that are stateless other than the implicit state of their endpoints.

'(), (void), and #f may also be present in action-trees: when the VM reaches such a value, it ignores it and continues with the next leaf in the tree.

For example, all of the following are valid action trees which will send messages 1, 2 and 3 in that order:

(list (send-message 1)

(send-message 2)

(send-message 3))

(list (list (send-message 1))

(cons (send-message 2) (cons '() (send-message 3))))

(cons (cons (send-message 1)

(send-message 2))

(list #f #f (send-message 3)))

Because #f and (void) are valid, ignored, members of an action-tree, and and when can be used to selectively include actions in an action-tree:

(list (first-action)

(when (condition?)

(optional-action))

(final-action))

(list (first-action)

(and (condition?)

(optional-action))

(final-action))

Finally, these inert placeholders can be used to represent "no action at all" in a transition:

(transition new-state) ; No action-trees at all

(transition new-state '())

(transition new-state (void))

(transition new-state #f)

For example,

(sequence-actions (transition 'x

(send-message (list 'message 0)))

(send-message (list 'message 1))

(send-message (list 'message 2))

(lambda (old-state)

(transition (cons 'y old-state)

(send-message (list 'message 3))))

(send-message (list 'message 4)))

produces the equivalent of

(transition (cons 'y 'x)

(send-message (list 'message 0))

(send-message (list 'message 1))

(send-message (list 'message 2))

(send-message (list 'message 3))

(send-message (list 'message 4)))

2.4 Creating endpoints

The primitive action that creates new endpoints is add-endpoint, but because endpoints are the most flexible and complex point of interaction between a process and its VM, a DSL, endpoint, streamlines endpoint setup.

For example, a minimal endpoint subscribing to all messages would be:

(endpoint #:subscriber ?)

or in Typed Racket, for a process with Integer as its process state type,

(endpoint: : Integer #:subscriber ?)

A minimal publishing endpoint would be:

(endpoint #:publisher ?)

(endpoint: : Integer #:publisher ?)

While topic patterns are ordinary Racket data with embedded ? wildcards (see Messages and Topics), all the other patterns in an endpoint are match-patterns. In particular note that ? is a wildcard in a topic pattern, while _ is a wildcard in a match-pattern.

2.4.1 Receiving messages

Supply one or more message-handler clauses to handle incoming message events (as distinct from presence- or absence-events).

The following endpoint subscribes to all messages, but only handles some of them:

(endpoint #:subscriber ?

['ping (send-message 'pong)]

['hello (list (send-message 'goodbye)

(quit))])

2.4.2 Action-only vs. State updates

If #:state occurs in an endpoint, or the maybe-typed-state-pattern occurs in an endpoint:, then all the handler-exprs in that endpoint are expected to return transition structures.

If not, however, the event handler expressions are expected to return plain ActionTrees.

This way, simple endpoints that do not need to examine the process state, and simply act in response to whichever event triggered them, can be written without the clutter of threading the process state value through the code.

For example, a simple endpoint could be written either as

(endpoint #:subscriber 'ping

['ping (send-message 'pong)])

or, explicitly accessing the endpoint’s process’s state,

(endpoint #:subscriber 'ping

#:state old-state

['ping (transition old-state

(send-message 'pong))])

2.4.3 Naming endpoints

Endpoint names can be used to update or delete endpoints.

If #:name is supplied, the given value is used as the name of the endpoint. If not, a fresh name is created. (At present, gensym is used.)

If #:let-name is supplied, the given identifier is bound in the handler-exprs to the name of the endpoint. If not, the name of the endpoint is inaccessible to the handler-exprs.

2.4.4 Handling presence and absence events

Other endpoints (in this or other processes) may have matching topics and complementary orientations to the current endpoint. When such endpoints come and go, presence and absence events are generated in the current endpoint.

By default, no actions are taken on such events, but #:on-presence and #:on-absence override this behaviour.

For example, say process A establishes the following endpoint:

(endpoint #:subscriber 'ping

#:on-presence (send-message 'pinger-arrived)

#:on-absence  (send-message 'pinger-departed)

['ping (send-message 'pong)])

Some time later, process B takes the following endpoint-establishing action:

(endpoint #:publisher 'ping

#:let-name ping-endpoint-name

#:on-presence

(list (endpoint #:subscriber 'pong

#:let-name pong-waiter-name

['pong (list (delete-endpoint ping-endpoint-name)

(delete-endpoint pong-waiter-name))])

(send-message 'ping)))

The sequence of events will be:

1. Process A’s #:on-presence handler will run, and the 'pinger-arrived message will be sent. At the same time,In the current implementation, one happens before the other, but it is nondeterministic which is run first. process B’s #:on-presence handler runs, installing a second endpoint and sending the 'ping message.

2. Process A’s endpoint receives the 'ping message, and sends the 'pong message.

3. Process B’s second endpoint receives the 'pong message, and deletes both of process B’s endpoints.

4. The #:on-absence handler in process A runs, sending the 'pinger-departed message.

One possible trace of messages in the VM containing processes A and B is

'pinger-arrived

'ping

'pong

'pinger-departed

By sending the 'ping message only once the #:on-presence handler has fired, process B ensures that someone is listening for pings.

This way, if process B starts before process A, then B will automatically wait until A is ready to receive ping requests before issuing any.

2.4.5 Exit reasons

If a #:reason pattern is supplied, then the exit reason supplied to the delete-endpoint or quit action that led to the absence-event is available to the endpoint’s #:on-absence handler expression.

2.4.6 Updating endpoints

If, when an endpoint is created, an existing endpoint with an equal? name is already present, then if the existing and to-be-added endpoints have exactly equal roles (meaning equal orientations, interest-types, and topic patterns), the handlers for the endpoint are updated without emitting presence or absence notifications.

This dubious feature can be used to avoid "glitching" of presence signals. A future release of this library will include better automatic support for avoiding such transients.

2.4.7 Who am I talking to?

If either #:role or any of #:peer-orientation, #:conversation, or #:peer-interest-type are supplied, the handler-exprs are given access to the role carried in the EndpointEvent that triggered them.

This role describes the intersection of interests between the current endpoint and the peer endpoint, and so can proxy for the identity of the other party. It is in a sense a description of the scope of the current conversation.

Using #:role allows a handler complete access to the role structure in the triggering event. It is more common however to simply use #:conversation to extract the role-topic alone, since it is seldom necessary to examine role-orientation (since it’s guaranteed to be complementary to the orientation of the current endpoint) or role-interest-type. If access to those parts is required, use #:peer-orientation and #:peer-interest-type.

2.5 Deleting endpoints

If reason is supplied, it is included in the corresponding action, and made available in any resulting absence-events.

2.6 Sending messages and feedback

2.7 Creating processes

If #:pid is supplied, the associated identifier is bound to the child process’s PID in both boot-expr and the parent’s k-expr.

Any supplied #:debug-name will be used in VM debug output. See also logging (MATRIX_LOG).

If #:parent is supplied, the associated k-expr will run in the parent process after the child process has been created. If the parent-state-pattern is also supplied, then k-expr must return a Transition; otherwise, it must return an ActionTree. Note that in Typed Racket, for type system reasons, spawn: requires ParentStateType to be supplied.

2.8 Exiting and killing processes

If reason is supplied, it is included in the corresponding action, and made available in any resulting absence-events.

Terminating the current process is as simple as:

(quit)

When a process raises an exception that it does not catch, its containing VM catches the exception and turns it into an implicit quit action. In that case, the reason will be the raised exception itself.

2.9 Cooperative scheduling

If pattern is supplied, k-expr should evaluate to a Transition; otherwise it should produce an ActionTree.

2.10 Creating nested VMs

If #:vm-pid is present, the corresponding identifier is bound in the boot-action expressions to the container-relative PID of the new VM itself. If #:boot-pid is present, however, the corresponding identifier is bound to the new-VM-relative PID of the primordial process in the new VM.

2.11 Relaying across layers

Marketplace’s actions can apply to either of those two networks. By default, actions apply to the VM of the acting process directly, but using at-meta-level (or at-meta-level: in typed code) to wrap an action level-shifts the action to make it apply at the level of the acting process’s VM’s container instead.

For example, wrapping an endpoint in at-meta-level adds a subscription to the VM’s container’s network. Instead of listening to sibling processes of the acting process, the new endpoint will listen to sibling processes of the acting process’s VM. In this example, the primordial process in the nested-vm creates an endpoint in the VM’s own network, the ground VM:

(nested-vm

(at-meta-level

(endpoint #:subscriber (tcp-channel ? (tcp-listener 5999) ?) ...)))

In this example, a new process is spawned as a sibling of the nested-vm rather than as a sibling of its primordial process:

(nested-vm

(at-meta-level

(spawn #:child (transition/no-state (send-message 'hello-world)))))

Compare to this example, which spawns a sibling of the nested-vm’s primordial process:

(nested-vm

(spawn #:child (transition/no-state (send-message 'hello-world))))