26 KiB
Syndicate Protocol
Actors that share a local scope can communicate directly. To communicate further afield, scopes are connected using relay actors. Relays allow indirect communication: distant entities can be addressed as if they were local.
Relays exchange Syndicate Protocol messages across a transport. A transport is the underlying medium connecting one relay to its counterparts on a given network. For example, a TLS-on-TCP/IP socket may connect a pair of relays to one another, or a UDP multicast socket may connect an entire group of relays across an ethernet.1
| TLS/TCP/IP socket |<-->|Relay| . . . || +-----+ +------------------------+ +-----+ | | | | | +-------------+ +-------------+
-->
## Transport requirements
Transports must
- be able to carry [Preserves](./glossary.md#preserves) values back and forth,
- be reliable and in-order,
- have a well-defined session lifecycle (created → connected → disconnected), and
- assure confidentiality, integrity and replay-resistance.
This document focuses primarily on point-to-point transports, discussing multicast and
in-memory variations briefly toward the end.
## Roles and session establishment
The protocol is completely symmetric, aside from [certain conventions detailed
below](#well-known-oids) about the entities available for use immediately upon connection
establishment. It is *not* a client/server protocol.
To begin a session on a newly-established point-to-point link, a relay simply starts sending
packets. Each peer starts the session with an empty entity reference map ([see
below](#membranes)) and making no assertions in either the outbound (on behalf of local
entities) or inbound (on behalf of the remote peer) directions.
## Packet definitions
```preserves-schema
Packet = Turn / Error / Extension .
Packets exchanged by relays are Preserves values defined using Preserves schema.
A packet may be a turn, an error, or an extension.
Packets are neither commands nor responses; they are events.
Extension packets
Extension = <<rec> @label any @fields [any ...]> .
An extension packet must be a Preserves record, but is otherwise unconstrained.
Handling. Peers MUST ignore extensions that they do not understand.2
Error packets
Error = <error @message string @detail any>.
Handling. An error packet describes something that went wrong on the other end of the connection. Error packets are primarily intended for debugging.
Receipt of an error packet denotes that the sender has terminated (crashed) and will not respond further; the connection will usually be closed shortly thereafter.
Error packets are optional: connections may simply be closed without comment.
Turn packets
Turn = [TurnEvent ...].
TurnEvent = [@oid Oid @event Event].
Event = Assert / Retract / Message / Sync .
Assert = <assert @assertion Assertion @handle Handle>.
Retract = <retract @handle Handle>.
Message = <message @body Assertion>.
Sync = <sync @peer #!#t>.
Assertion = any .
Handle = int .
Oid = int .
A Turn is the most important packet variant. It directly reflects the
SAM notion of a turn.
Handling. Each Turn carries events to be delivered to
entities residing at the receiving end of the transport.
The assertion fields of Assert events and the body fields of Message events may contain
any Preserves value, including embedded entity references. On the wire, these will always be
formatted as described below. Upon receipt of a Turn, embedded
references are first mapped to internal references. Prior to transmission, internal references
are mapped to their external form. The mapping procedure to follow is detailed
below.
After reference rewriting is complete, the sequence of TurnEvents is examined. The
OID in each TurnEvent selects an entity known to the recipient. Each
Event is either publication of an assertion, retraction of a previously-published assertion,
delivery of a single message, or a synchronization event.
In the case that the receiving party is structured internally using the SAM, it is important to
preserve turn boundaries. Since turn boundaries are a per-actor concept,
but a Turn mentions only entities, the receiver must map entities to actors, group
TurnEvents into per-actor queues, and deliver those queues to each actor in a single SAM turn
for each actor.
The Handles used to refer to published assertions MUST be unique within the scope of the
transport connection.
Capabilities on the wire
Packets sent and received on a point-to-point transport frequently include embedded references. These references denote capabilities for interacting with some entity.
For example, assertion of a capability-bearing record could appear as the following Event:
<assert <please-reply-to #![0 555]>>
The #![0 555] is concrete Preserves text syntax for
an embedded (#!) value ([0 555]).
In the Syndicate Protocol, these embedded values MUST conform to the WireRef
schema:3
WireRef = @mine [0 @oid Oid] / @yours [1 @oid Oid @attenuation Caveat ...].
Oid = int .
The mine variant denotes capability references managed by the sender of a given packet; the
yours variant, the receiver of the packet. Accordingly, if a relay receives a packet
mentioning #![0 555], it will later use #![1 555] if it needs to send a packet to refer to
that same entity.
Attenuation of authority
A yours-variant capability may include a request4 to impose
additional conditions on the receiver's use of its own capability, known as an
attenuation of the capability's authority.
An attenuation is a chain of Caveats.5 A Caveat acts as a
function that, given a Preserves value representing an assertion or message body, yields either
a possibly-rewritten value, or no value at all. In the latter case, the value has been
rejected. In the former case, the rewritten value is used as input to the next Caveat in
the chain, or as the final assertion or message body for delivery to the entity backing the
capability.
The chain of Caveats in an attenuation is written down in reverse order: newer Caveats
are appended to the sequence, and each Caveat's output is fed into the input of the next
leftward Caveat in the sequence. If no Caveats are present, the capability is unattenuated,
and inputs are passed through to the backing capability unmodified.
Caveat = Rewrite / Alts .
Rewrite = <rewrite @pattern Pattern @template Template>.
Alts = <or @alternatives [Rewrite ...]>.
A Caveat can be either a single Rewrite or a sequence of alternative possible rewrites,
tried in left-to-right order until one of them accepts the input or there are none left to try.
(A single Rewrite R is equivalent to <or [R]>.)
A Rewrite applies its Pattern to the input to the Caveat. If it matches, the bindings
captured by the pattern are gathered together and used in instantiation of the Rewrite's
Template, yielding the output from the Caveat. If the pattern does not match, the Rewrite
has rejected the input, and other alternatives are tried until none remain, at which point
the whole Caveat has rejected the input and processing of the triggering event stops.
Patterns
A Pattern within a rewrite can be any of the following variants:
Pattern = PDiscard / PAtom / PEmbedded / PBind / PAnd / PNot / Lit / PCompound .
PDiscard matches any value:
PDiscard = <_>.
PAtom requires that a matched value be a boolean, a single- or double-precision float, an
integer, a string, a binary blob, or a symbol, respectively:
PAtom = =Boolean / =Float / =Double / =SignedInteger / =String / =ByteString / =Symbol .
PEmbedded requires that a matched value be an embedded capability:
PEmbedded = =Embedded .
PBind first captures the matched value, adding it to the bindings vector, and then applies
the nested pattern. If the subpattern matches, the PBind succeeds; otherwise, it fails:
PBind = <bind @pattern Pattern>.
PAnd is a conjunction of patterns; every pattern in patterns must match for the PAnd to
match:
PAnd = <and @patterns [Pattern ...]>.
PNot is a pattern negation: if pattern matches, the PNot fails to match, and vice
versa. It is an error for pattern to include any PBind subpatterns.
PNot = <not @pattern Pattern>.
Lit is an exact match pattern. If the matched value is exactly equal to value (according to
Preserves' own built-in equivalence relation), the match succeeds; otherwise, it fails:
Lit = <lit @value any>.
Finally, PCompound patterns match compound data structures. The rec variant demands that a
matched value be a record, with label exactly equal to label and fields one-for-one matching
the Patterns in fields; the arr variant demands a sequence, with each element matching
the corresponding element of items; and dict demands a dictionary having at least entries
named by the keys of the entries dictionary, each matching the corresponding Pattern.
PCompound =
/ @rec <rec @label any @fields [Pattern ...]>
/ @arr <arr @items [Pattern ...]>
/ @dict <dict @entries { any: Pattern ...:... }> .
Bindings
Bindings resulting from matching are stored as a sequence of values.
During matching, when a PBind pattern is seen, the matcher first appends the matched value
to the binding sequence and then recurses on the nested subpattern. This makes binding
indexes appear in left-to-right order as a Pattern is read.
For example, given the pattern <bind <arr [<bind <_>>, <bind <_>>]>> and the matched value
[1 2], the resulting captured values will be, in order, [1 2], 1, and 2.
Templates
A Template within a rewrite produces a concrete Preserves value when instantiated with a
vector of captured binding values. Template instantiation may fail, yielding no value.
A given Template may be any of the following variants:
Template = TAttenuate / TRef / Lit / TCompound .
TAttenuate first instantiates the sub-template. If it yields a value, and if that value is
an embedded reference (i.e. a capability), the Caveats in attenuation are appended to the
(possibly-empty) sequence of Caveats already present in the embedded capability. The
resulting possibly-attenuated capability is the final result of instantiation of the
TAttenuate.
TAttenuate = <attenuate @template Template @attenuation [Caveat ...]>.
TRef retrieves the bindingth (0-based) index into the bindings vector, yielding the
associated captured value as the result of instantiation. It is an error if binding is less
than zero, or greater than or equal to the number of bindings in the bindings vector.
TRef = <ref @binding int>.
Lit (the same definition as used in the grammar for Pattern above) instantiates to exactly
its value argument:
Lit = <lit @value any>.
Finally, TCompound instantiates to compound data. The rec variant produces a record with
the given label and fields; arr produces an array; and dict a dictionary:
TCompound =
/ @rec <rec @label any @fields [Template ...]>
/ @arr <arr @items [Template ...]>
/ @dict <dict @entries { any: Template ...:... }> .
Validity of Caveats
The above definitions imply some validity constraints on Caveats.
-
All
TRefs must be bound: the index referred to must relate to the index associated with somePBindin the pattern corresponding to the template. -
Binding under negation is forbidden: a
patternwithin aPNotmay not include anyPBindconstructors. -
The value produced by instantiation of
templatewithin aTAttenuatemust be an embedded reference (a capability).
Implementations MUST enforce these constraints (either statically or dynamically).
Membranes
In order to correctly map between embedded references on the wire and entity references local to the relay, the relay maintains two stateful objects called membranes. A membrane is a bidirectional mapping between OID and relay-internal entity pointer.
-
The import membrane connects OIDs managed by the remote peer to local relay entities which proxy access to an "imported" remote entity.
-
The export membrane connects OIDs managed by the local peer to any local "exported" entities accessible to the peer.
|
|
Export Membrane | Import Membrane
|
+-+ | +-+
Pointer | | ID | ID | |
0x1234 <-+-+-> "my 7" | "your 7"<-+-+-> 0x9abc
| | | | |
^ | | ^ | ^ | | ^
| | | | -+- | | | |
V | | | | | | V
/------\ | | \-------------/ | | /------\
|Entity| | | | | |Relay |<-- ...
\------/ | | | | |Entity|
0x1234 | | --------> | | \------/
=-------------+-+---= packets | | 0x9abc
0x462e | | <-------- =---+-+-------------=
/------\ | | | | 0xa043
|Relay | | | | | /------\
... -->|Entity| | | /-------------\ | | |Entity|
\------/ | | | | | | \------/
^ | | | -+- | | | ^
| | | | | | | | |
V | | V | V | | V
| | | | |
0x462e <-+-+->"your 3" | "my 3" <-+-+-> 0xa043
Pointer | | ID | ID | | Pointer
+-+ | +-+
|
Import Membrane | Export Membrane
|
|
---------------------------------------+
|
Membrane |
----+---- |
/--\ | | |
|A1|<----+ Pointer | ID +-+-------------transport---
\--/ | |
+---0x7f1065218700<-+-> 7
| <---- from remote
/--\ | peer
|A2|<-----0x7f1065229780<-+-> 11
\--/ |
| ----> to remote
+--0x7f10652fe7c0<-+-> 13 peer
/--\ | |
|A3|<----+ | +-+-------------------------
\--/ | |
|
|
----------------------------------------+
+----------------+ (2) +-------+ | +---------+ | | ^ ->|Dataspace| | | | +---------+ | | V ^ ^ ^ (1) | +----------------+ TCP/IP +-------+ | | | | |Remote Syndicate|<----------->| Relay |<----+-/ | | | |server/dataspace| (3) +-------+ | | | | +----------------+ | V V V | | +-------+ +-------+ | | | Relay | | Relay | | +----------+-------+-+-------+--+ ^ ^ LAN multicast (4) | | UNIX ... <-------/---------/---------/---------/ | socket | | | | (5) v v v v +-------+ +-------+ +-------+ +-------+ |. . . | |. . . | |. . . | |. . . | +-------+ +-------+ +-------+ +-------+
In the diagram above, networks (scopes) 1 and 4 are *multicast*, while networks 2, 3 and 5 are
*point-to-point*. Four relays bridge scope 1 to scopes 2 through 5. Within each scope, peers
are able to interact with each other directly. Each point-to-point scope contains exactly two
peers.
-->
## Client and server roles
## Well-known OIDs
OID 0, initial ref, initial oid
## Security considerations
### Secrecy
### Privacy
## Specific transport mappings
TCP/IP
TLS TCP/IP
WebSockets
## Other kinds of medium
Multicast/broadcast, in-memory
## Appendix: Complete schema of the protocol
The following is a consolidated form of the definitions from the text above.
### Protocol packets
The authoritative
version of this schema is
[`[syndicate-protocols]/schemas/protocol.prs`](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/branch/main/schemas/protocol.prs).
```preserves-schema
version 1 .
Packet = Turn / Error / Extension .
Extension = <<rec> @label any @fields [any ...]> .
Error = <error @message string @detail any>.
Assertion = any .
Handle = int .
Event = Assert / Retract / Message / Sync .
Oid = int .
Turn = [TurnEvent ...].
TurnEvent = [@oid Oid @event Event].
Assert = <assert @assertion Assertion @handle Handle>.
Retract = <retract @handle Handle>.
Message = <message @body Assertion>.
Sync = <sync @peer #!#t>.
Capabilities, WireRefs, and attenuations
The authoritative version of this schema is
[syndicate-protocols]/schemas/sturdy.prs.
version 1 .
Attenuation = [Caveat ...].
Caveat = Rewrite / Alts .
Rewrite = <rewrite @pattern Pattern @template Template>.
Alts = <or @alternatives [Rewrite ...]>.
Oid = int .
WireRef = @mine [0 @oid Oid] / @yours [1 @oid Oid @attenuation Caveat ...].
Lit = <lit @value any>.
Pattern = PDiscard / PAtom / PEmbedded / PBind / PAnd / PNot / Lit / PCompound .
PDiscard = <_>.
PAtom = =Boolean / =Float / =Double / =SignedInteger / =String / =ByteString / =Symbol .
PEmbedded = =Embedded .
PBind = <bind @pattern Pattern>.
PAnd = <and @patterns [Pattern ...]>.
PNot = <not @pattern Pattern>.
PCompound =
/ @rec <rec @label any @fields [Pattern ...]>
/ @arr <arr @items [Pattern ...]>
/ @dict <dict @entries { any: Pattern ...:... }> .
Template = TAttenuate / TRef / Lit / TCompound .
TAttenuate = <attenuate @template Template @attenuation Attenuation>.
TRef = <ref @binding int>.
TCompound =
/ @rec <rec @label any @fields [Template ...]>
/ @arr <arr @items [Template ...]>
/ @dict <dict @entries { any: Template ...:... }> .
Appendix: Pseudocode for attenuation, pattern matching, and template instantiation
Attenuation
def attenuate(attenuation, value):
for caveat in reversed(attenuation):
value = applyCaveat(caveat, value)
if value is None:
return None
return value
def applyCaveat(caveat, value):
if caveat is 'Alts' variant:
for rewrite in caveat.alternatives:
possibleResult = tryRewrite(rewrite, value);
if possibleResult is not None:
return possibleResult
return None
if caveat is 'Rewrite' variant:
return tryRewrite(caveat, value)
def tryRewrite(rewrite, value):
bindings = applyPattern(rewrite.pattern, value)
if bindings is None:
return None
else:
return instantiateTemplate(rewrite.template, bindings)
Pattern matching
def match(pattern, value, bindings):
if pattern is 'PDiscard' variant:
return True
if pattern is 'PAtom' variant:
return True if value is of the appropriate atomic class else False
if pattern is 'PEmbedded' variant:
return True if value is a capability else False
if pattern is 'PBind' variant:
append value to bindings
return match(pattern.pattern, value, bindings)
if pattern is 'PAnd' variant:
for p in pattern.patterns:
if not match(p, value, bindings):
return False
return True
if pattern is 'PNot' variant:
return False if match(pattern.pattern, value, bindings) else True
if pattern is 'Lit' variant:
return (pattern.value == value)
if pattern is 'PCompound' variant:
if pattern is 'rec' variant:
if value is not a record: return False
if value.label is not equal to pattern.label: return False
if value.fields.length is not equal to pattern.fields.length: return False
for i in [0 .. pattern.fields.length):
if not match(pattern.fields[i], value.fields[i], bindings):
return False
return True
if pattern is 'arr' variant:
if value is not a sequence: return False
if value.length is not equal to pattern.items.length: return False
for i in [0 .. pattern.items.length):
if not match(pattern.items[i], value[i], bindings):
return False
return True
if pattern is 'dict' variant:
if value is not a dictionary: return False
for k in keys of pattern.entries:
if k not in keys of value: return False
if not match(pattern.entries[k], value[k], bindings):
return False
return True
Template instantiation
def instantiate(template, bindings):
if template is 'TAttenuate' variant:
c = instantiate(template.template, bindings)
if c is not a capability: raise an exception
c′ = c with the caveats in template.attenuation appended to the existing
attenuation in c
return c′
if template is 'TRef' variant:
if 0 ≤ template.binding < bindings.length:
return bindings[template.binding]
else:
raise an exception
if template is 'Lit' variant:
return template.value
if template is 'TCompound' variant:
if template is 'rec' variant:
return Record(label=template.label,
fields=[instantiate(t, bindings) for t in template.fields])
if template is 'arr' variant:
return [instantiate(t, bindings) for t in template.items]
if template is 'dict' variant:
result = {}
for k in keys of template.entries:
result[k] = instantiate(template.entries[k], bindings)
return result
Notes
-
In fact, it makes perfect sense to run the relay protocol between actors that are already connected in some scope: this is like running a VPN, tunnelling IP over IP. A variation of the Syndicate Protocol like this gives federated dataspaces. ↩︎
-
This specification does not define any extensions, but future revisions could, for example, use extensions to perform version-negotiation. Another potential future use could be to propagate provenance information for tracing/debugging. ↩︎
-
The syntax for
WireRefs is slightly silly, using tuples as quasi-records with0and1acting as quasi-labels. It would probably be better to use real records, like<my @oid Oid>and<yours @oid Oid @attenuation [Caveat ...]>. Pros: less cryptic. Cons: slightly more verbose on the wire. TODO: should we revise the spec in this regard? ↩︎ -
Such conditions can only ever be requests: after all, every
yours-capability is already completely accessible to the recipient of the packet. Similarly, it does not make sense to include an attenuation description on amy-capability. However, in every case, if a party wishes to enforce an attenuation on amy- oryours-capability, it may record the attenuation against the underlying capability internally, issuing to its peers a freshmy-capability denoting the attenuated capability. ↩︎ -
This terminology, "caveat", is lifted from the excellent paper on Macaroons, where it is used to describe a more general mechanism. Future versions of this specification may opt to include some of this generality. ↩︎