syndicate-2017/racket/syndicate/scribblings/syndicate.scrbl

#lang scribble/manual

@(require (for-label (except-in racket process field)
            syndicate/actor))

@title{Dataspace Programming with Syndicate}


@defmodule[syndicate/actor]

@section{Overview}

Syndicate is an actor language where all communication occurs through a tightly
controlled shared memory, dubbed the @emph{dataspace}. The values in the
dataspace are called @emph{assertions}, representing the information that the
actors in the system are currently sharing with each other. Assertions are
@emph{read-only} and @emph{owned} by the actor that entered them into the
dataspace. Only the originating actor has permission to withdraw an assertion.
Assertions are linked to the lifetime of their actor, and are withdrawn from the
dataspace when that actor exits, either normally or via exception.

To respond to assertions in the dataspace, an actor expresses an @emph{interest}
in the shape of assertions it wishes to receive. An interest is an assertion
constructed with @racket[observe] and wildcards where the actor wishes to
receive any matching assertion. When an actor makes an assertion of interest,
the dataspace dispatches the set of all matching assertions to that actor.
Moreover, the dataspace keeps the actor @emph{up-to-date}, informing it when a
new assertion appears matching its interest, as well as when a matching
assertion disappears from the dataspace. Thus, dataspaces implement a form of
publish/subscribe communication.

@;{would be nice to link pub/sub}

In addition to assertions, dataspaces support instantaneous @racket[message]
broadcast. At the time a message is sent, all actors with a matching interest
receive notification.

In response to an event, that is, a message broadcast or assertion
appearance/disappearance matching an expressed interest, a Syndicate actor may
take any of the following actions:
@itemlist[
  @item{Updating its internal state;}
  @item{Making or withdrawing assertions;}
  @item{Sending broadcast messages;}
  @item{Spawning additional actors;}
  @item{Exiting;}
  @item{Or any combination of these.}
]

Thus, each individual Syndicate actor has three fudamental concerns:

@itemlist[
  @item{Defining local state and updating it in response to events;}
  @item{Publishing aspects of local state/knowledge as assertions; and}
  @item{Reacting to relevant assertions and messages.}
]

Each concern is addressed by a separate language construct, which are
collectively dubbed @emph{endpoints}:

@itemlist[
  @item{The @racket[field]s of an actor hold its state;}
  @item{An actor publishes information using @racket[assert]; and}
  @item{An event-handler endpoint uses @racket[on] to define reactions to
  particular messages and assertions.}
]

Endpoints are tied together via @emph{dataflow}. Thus, the assertions of an
actor automatically reflect the current value of its fields.

Implementing an actor's role in a particular conversation typically involves
some combination of these behaviors; a @emph{facet} is a collection of related
endpoints constituting the actor's participation in a particular conversation.

Each actor starts with a single facet, and may add new facets or terminate
current ones in response to events. The facets of an actor form a tree, where
the parent of a particular facet is the facet in which it was created. The tree
structure affects facet shutdown; terminating a facet also terminates all of its
descendants.

To recap: an actor is a tree of facets, each of which comprises of a collection
of endpoints.

@section{Programming Syndicate Actors with Facets}

Code within Syndicate actors executes in one of two contexts:
@itemlist[
  @item{The @emph{endpoint-installation} context occurs during the creation of a
  new facet, when all of its endpoints are created.}
  @item{The @emph{script} context occurs during the execution of event handlers,
  and permits creating/terminating facets, sending messages, and spawning
  actors.}
]

The actions permitted by the two contexts are mutually exclusive, and trying to
perform an action in the wrong context will give rise to a run-time
@racket[error].

Within the following descriptions, we use @emph{EI} as a shorthand for
expressions that execute in an endpoint-installation context and @emph{S} for
expressions in a script context.

@subsection{Script Actions: Starting and Stopping Actors and Facets}

@defform[(spawn EI ...+)]{
Spawn an actor with a single inital facet whose endpoints are installed by
@racket[EI]. That is, there is an implicit @racket[react] around @racket[EI
...]. Allowed within a script and module-top-level.}

@defform[(react EI ...+)]{
Create a new facet in the current actor whose endpoints are the result of
executing @racket[EI ...]. Allowed within a script.
}

@defform[(stop-facet fid S ...)
         #:contracts ([fid facet-id?])]{
Terminate the facet with ID @racket[fid], as well as all of its children.
Allowed within a script.

The optional script actions @racket[S ...] function like a continuation. They
run @emph{after} the facet and all of its children finish shutting down, i.e.
after all @racket[stop] handlers have executed. Moreover, @racket[S ...] runs in
the context of the @emph{parent} of @racket[fid]. Thus, any facet created by the
script survives termination and will have @racket[fid]'s parent as its own
parent.

Note that @racket[fid] must be an ancestor of the current facet.
}

@defform[(stop-current-facet S ...)]{
Stop the currently running facet; equivalent to @racket[(stop-facet
(current-facet-id) S ...)]. Allowed within a script.
}

@defproc[(current-facet-id) facet-id?]{
Retrieves the ID of the currently running facet.
}

@defproc[(send! [v any/c])
         void?]{
Sends a @racket[message] with body @racket[v].}

@subsection{Installing Endpoints}

@defform[(field [x init-expr maybe-contract] ...+)
         #:grammar
         [(maybe-contract (code:line)
                          (code:line #:contract in)
                          (code:line #:contract in out))]]{
Define fields for the current facet. Each @racket[x] is bound to a handle
function: calling @racket[(x)] retrieves the current value, while @racket[(x v)]
sets the field to @racket[v].

Fields may optionally have a contract; the @racket[in] contract is applied when
writing to a field, while the (optional) @racket[out] contract applies when
reading a value from a field.

Allowed within an endpoint installation context.
}

@defform[(assert maybe-pred exp)
         #:grammar
         [(maybe-pred (code:line)
                      (code:line #:when pred))]
         #:contracts ([pred boolean?])]{
Make the assertion @racket[exp] while the enclosing facet is active. Publishing
the assertion can be made conditional on a boolean expression by supplying a
@racket[#:when] predicate, in which case the assertion is made only when
@racket[pred] evaluates to @racket[#t].

If the expression @racket[exp] refers to any fields, then the assertion created
by the endpoint is automatically kept up-to-date each time any of those fields
is updated. More specifically, the will issue a patch retracting the assertion
of the previous value, replacing it with the results of reevaluating
@racket[exp] with the current values of each field.

Allowed within an endpoint installation context.
}

@defform[#:literals (message asserted retracted _ $ ?)
         (on event-description
             S ...+)
         
         #:grammar
         [(event-description (code:line message pattern)
                             (code:line asserted pattern)
                             (code:line retracted pattern))
          (pattern (code:line _)
                   (code:line $id)
                   (code:line ($ id pattern))
                   (code:line (? pred pattern))
                   (code:line (ctor pattern ...))
                   (code:line expr))]]{
Creates an event handler endpoint that responds to the event specified by
@racket[event-description]. Executes the body @racket[S ...] for each matching
event, with any pattern variables bound to their matched value. Additionally,
the actor will make an assertion of interest in events that could match
@racket[event-description].

Allowed within an endpoint installation context.

Event descriptions have one of the following forms:
@itemlist[
  @item{@racket[(message pattern)] activates when a message is received with a
    body matching @racket[pat].}

  @item{@racket[(asserted pattern)] activates when a patch is received with an
    added assertion matching @racket[pattern]. Additionally, if the actor has
    @emph{already} received a patch with matching assertions, which can occur if
    multiple facets in a single actor have overlapping interests, then the
    endpoint will match those assertions upon facet start up.}

  @item{@racket[(retracted pat)] is similar to @racket[asserted], but for
    assertions withdrawn in a patch.}

  @;{@item{@racket[(rising-edge expr)] activates when @racket[expr] evaluates to
    anything besides @racket[#f] (having previously evaluated to @racket[#f]). The
    condition is checked after each received event.}}
]

While patterns have the following meanings:
@itemlist[
  @item{@racket[_] matches anything.}

  @item{@racket[$id] matches anything and binds the value to @racket[id].}

  @item{@racket[($ id pattern)] matches values that match @racket[pattern] and
    binds the value to @racket[id].}

  @item{@racket[(? pred pattern)] matches values where @racket[(pred val)] is not
     @racket[#f] and that match @racket[pattern].}

  @item{@racket[(ctor pat ...)] matches values built by applying the constructor
    @racket[ctor] to values matching @racket[pat ...]. @racket[ctor] is usually
    a @racket[struct] name.}

  @item{@racket[expr] patterns match values that are @racket[equal?] to
    @racket[expr].}
]
}

@defform[(during pattern EI ...+)]{
Engage in behavior for the duration of a matching assertion. Roughly equivalent
to:

@racketblock[
(on (asserted pattern)
  (react
    EI ...
    (on (retracted inst-pattern)
        (stop-current-facet))))]

where @racket[inst-pattern] is the @racket[pattern] with variables instantiated
to their matching values.

Allowed within an endpoint installation context.
}

@subsection{Handling Facet Startup and Shutdown}

In addition to external events, such as assertion (dis)appearance and message
broadcast, facets can react to their own startup and shutdown. This provides a
handy way to perform initialization, cleanup, as well as setting up and tearing
down resources.

@defform[(on-start S ...)]{
Perform the script actions @racket[S ...] upon facet startup.

Allowed within an endpoint installation context.
}

@defform[(on-stop S ...)]{
Perform the script actions @racket[S ...] upon facet shutdown.

The script @racket[S ...] differs from that of @racket[stop-facet] in that it
executes in the context of the terminating facet, not its parent. Thus, any
facets created in @racket[S ...] will start up and then immediately shut down.

Allowed within an endpoint installation context.
}

Note that a single facet may have any number of @racket[on-start] and
@racket[on-stop] handlers, which do not compete with each other. That is, each
@racket[on-start] handler runs during facet startup and, likewise, each
@racket[on-stop] during facet shutdown.

@subsection{Streaming Query Fields}

Syndicate actors often aggregate information about current assertions as part of
their local state, that is, in a @racket[field]. Since these patterns are
exceedingly common, Syndicate provides a number of forms for defining fields
that behave as streaming queries over the assertions in the dataspace.

@defform[(define/query-set name pattern expr maybe-on-add maybe-on-remove)
         #:grammar
         [(maybe-on-add (code:line)
                        (code:line #:on-add on-add-expr))
          (maybe-on-remove (code:line)
                           (code:line #:on-remove on-remove-expr))]]{
Define a @racket[field] called @racket[name] that is the @racket[set] of values
extracted from assertions matching @racket[pattern]. Each value is extracted
from a matching assertion by evaluating @racket[expr], which may refer to
variables bound by @racket[pattern].

The query set expands to roughly the following code:
@racketblock[
(begin
  (field [name (set)])
  (on (asserted pattern)
      (name (set-add (name) expr)))
  (on (retracted pattern)
      (name (set-remove (name) expr))))]
    
The optional @racket[on-add-expr] is performed inside the @racket[on asserted]
handler, while @racket[on-remove-expr] runs in the @racket[on retracted]
handler.

Allowed within an endpoint installation context.
}

@defform[(define/query-value name absent-expr pattern expr
                               maybe-on-add
                               maybe-on-remove)
         #:grammar
         [(maybe-on-add (code:line)
                        (code:line #:on-add on-add-expr))
          (maybe-on-remove (code:line)
                           (code:line #:on-remove on-remove-expr))]]{
Define a @racket[field] called @racket[name] whose value is based on the
presence of an assertion matching @racket[pattern] in the dataspace. When such
an assertion is present, the value of the @racket[name] field is the result of
evaluating @racket[expr], which may refer to @racket[pattern]. When no such
assertion exists, including initially, the value of @racket[name] is the result
of @racket[absent-expr].

The optional @racket[maybe-on-add] and @racket[maybe-on-expr] behave the same
way they do for @racket[define/query-set].

Allowed within an endpoint installation context.
}

@subsection{Generalizing Dataflow}

Talk about begin/dataflow here.

@subsection{Generalizing Actor-Internal Communication}

Talk about internal assertions and messages.

@subsection{Nesting Dataspaces}

Nested dataspaces, inbound and outbound assertions.

@section{Interacting with the Outside World}

ground dataspace, drivers, etc.

@section{Actors with an Agenda}

Here we talk about @racket[spawn*] and @racket[react/suspend].

@section{Odds and Ends}

@defform[(dataspace I ...)]{
Spawns a dataspace as a child of the dataspace enclosing the executing actor. The
new dataspace executes each instantaneous action @racket[I].}

@defproc[(assert! [v any/c]
                  [#:meta-level level natural-number/c 0])
         void?]{
Asserts the value of @racket[v] until either explicitly retracted via
@racket[retract!] or the immediately enclosing actor exits. @racket[level]
specifies which dataspace the assertion should be made, in terms of relative
distance from the dataspace containing the enclosing actor.}

@defproc[(retract! [v any/c]
                   [#:meta-level level natural-number/c 0])
         void?]{
Retracts any assertions made by the immediately enclosing actor at
@racket[level] dataspaces above the enclosing dataspace of the form @racket[v].}

@defform[(state maybe-init (maybe-bindings O ...) ([E I ...] ...))
         #:grammar
         [(maybe-init (code:line)
                      (code:line #:init [I ...]))
          (maybe-bindings (code:line)
                          (code:line #:collect ([id init] ...)))]
         #:contracts ([id identifier?])]{
Spawns a new actor with ongoing behaviors @racket[O ...] that runs until a
termination event is detected.

The optional @racket[#:init [I ...]] provides a sequence of initialization
actions. The initial actions are executed before the ongoing behaviors begin but
after the interests of the state actor are established.

The optional @racket[#:collect [(id init) ...]] clause introduces bindings that
are visible within the body of the state actor. Each binding @racket[id] is
initialized to the corresponding @racket[init] expression. The bindings are
updated when an ongoing behavior executes an instantaneous event, such as the
result of an @racket[on] behavior. The new bindings are in the form of a
@racket[values] form, with the new values in the same order and number as in the
@racket[#:collect].

The ongoing behaviors @racket[O ...] are run simultaneously until the state
actor exits.

Each @racket[[E I ...]] specifies a termination event @racket[E] of the actor.
When a termination event @racket[E] activates, the corresponding @racket[I]s are
executed. The state actor then exits, with the same result of the final
@racket[I] action.}

@defform[(until E
                maybe-init
                maybe-bindings
                maybe-done
                O ...)
         #:grammar
         [(maybe-init (code:line)
                      (code:line #:init [I ...]))
          (maybe-bindings (code:line)
                          (code:line #:collect ([id init] ...)))
          (maybe-done (code:line)
                      (code:line #:done [I ...]))]
         #:contracts ([id identifier?])]{
An @racket[until] behavior corresponds to a @racket[state] behavior with only
one termination event, given by @racket[E]. The final result of the
@racket[until] behavior is the values of the @racket[#:collect] bindings in
scope from any parent actors followed by the final values of the @racket[until]
actor's bindings. The actions in a @racket[#:done] clause are executed after the
termination event but before the @racket[until] actor exits.}

@defform[(forever maybe-init
           maybe-bindings
           O ...)
         #:grammar
         [(maybe-init (code:line)
                      (code:line #:init [I ...]))
          (maybe-bindings (code:line)
                          (code:line #:collect ([id init] ...)))]
         #:contracts ([id identifier?])]{
The @racket[forever] behavior is analogous to a @racket[state] form with no
termination events.}