diff --git a/src/glossary.md b/src/glossary.md index 02be59e..4f7cef7 100644 --- a/src/glossary.md +++ b/src/glossary.md @@ -307,7 +307,7 @@ language, the Preserves concept of an *embedded value* is used in the SAM to rel portions of a datum referring to SAM entities. Concretely, in [Preserves text -syntax](https://preserves.dev/preserves.html#textual-syntax), embedded values +syntax](https://preserves.dev/preserves-text.html), embedded values appear prepended with `#!`. In messages transferred across links using the [Syndicate network protocol][], references might appear as `#![0 123]`, `#![1 555]`, etc. etc. diff --git a/src/guide/preserves.md b/src/guide/preserves.md index 47c34bd..9dff7c5 100644 --- a/src/guide/preserves.md +++ b/src/guide/preserves.md @@ -60,11 +60,11 @@ automatically, losslessly translatable from one syntax to another because Preser are syntax-independent. The core Preserves specification defines a text-based, human-readable, JSON-like syntax, that -is a syntactic superset of JSON, and a completely equivalent compact binary syntax, crucial to -the definition of [canonical form](#canonical-form) for Preserves values.[^syrup] +is a syntactic superset of JSON, and a completely equivalent compact machine-oriented syntax, +crucial to the definition of [canonical form](#canonical-form) for Preserves values.[^syrup] Here are a few example values, written using the text syntax (see [the -specification](https://preserves.dev/preserves.html#textual-syntax) for the +specification](https://preserves.dev/preserves-text.html) for the grammar): Boolean : #t #f @@ -84,10 +84,10 @@ Commas are optional in sequences, sets, and dictionaries. ### Canonical form -Every Preserves value can be serialized into a *canonical form* using the [binary -syntax](https://preserves.dev/preserves.html#compact-binary-syntax) along with -[a few simple rules](https://preserves.dev/canonical-binary.html) about -serialization ordering of elements in sets and keys in dictionaries. +Every Preserves value can be serialized into a *canonical form* using the [machine-oriented +syntax](https://preserves.dev/preserves-binary.html) along with [a few simple +rules](https://preserves.dev/canonical-binary.html) about serialization ordering of elements in +sets and keys in dictionaries. Having a canonical form means that, for example, a cryptographic hash of a value's canonical serialization can be used as a unique fingerprint for the value. diff --git a/src/operation/builtin/gatekeeper.md b/src/operation/builtin/gatekeeper.md index 775f9dc..57b060f 100644 --- a/src/operation/builtin/gatekeeper.md +++ b/src/operation/builtin/gatekeeper.md @@ -39,15 +39,15 @@ The following definitions are taken from the schema. For further detail, see the [reference](../../protocols/syndicate/gatekeeper.md). ```preserves-schema -SturdyRef = . +SturdyRef = . ``` Within a `ref` record, the `oid` field is a free-form value that the targeted service chooses to name itself. The `sig` is an iterated keyed-HMAC construction, just as in [macaroons][]. First, the service's secret key is used to key an HMAC of the `oid`. Then, the result is used -to key an HMAC of the first `Attenuation` in `caveatChain`. Each `Attenuation`'s HMAC becomes -the key for the next in the `caveatChain`. The final result is equal to the `sig` field in a -valid sturdyref. +to key an HMAC of the ([canonical form](../../guide/preserves.md#canonical-form) of the) first +`Caveat` in `caveatChain`. Each `Caveat`'s HMAC becomes the key for the next in the +`caveatChain`. The final result is equal to the `sig` field in a valid sturdyref. ### Attenuation of authority @@ -56,19 +56,13 @@ sturdyref, the `caveatChain` is used to [attenuate](../../glossary.md#attenuatio authority denoted by the sturdyref by filtering and/or rewriting assertion and message bodies. The `caveatChain` is run *right to left*, with newer rewrites-and-filters at the right-hand end of the chain and older ones at the left-hand end. Of course, an empty `caveatChain` is an -unattenuated reference. +unattenuated reference. The structure and interpretation of `Caveat`s is described fully in +[the relevant section of the Syndicate network protocol +specification](../../protocol.md#attenuation-of-authority). -```preserves-schema -Attenuation = [Caveat ...]. -``` - -Each individual `Attenuation` in a `caveatChain` is a sequence of `Caveat`s. The term "caveat" -is shamelessly taken from [macaroons][], though our caveats presently embody only what in the -Macaroons paper are called "first-party caveats" over assertion structure; future versions of -the server may add "third-party caveats" and other, richer, predicates over assertions. - -Each `Attenuation`'s `Caveat`s are run in *right to left* order. The structure and -interpretation of `Caveat`s is described fully in [the relevant section of the Syndicate -network protocol specification](../../protocol.md#attenuation-of-authority). +The term "caveat" is shamelessly taken from [macaroons][], though our caveats presently embody +only what in the Macaroons paper are called "first-party caveats" over assertion structure; +future versions of the server may add "third-party caveats" and other, richer, predicates over +assertions. [Macaroons]: ../../glossary.md#macaroon diff --git a/src/operation/scripting.md b/src/operation/scripting.md index c06c8fa..f1d466a 100644 --- a/src/operation/scripting.md +++ b/src/operation/scripting.md @@ -31,13 +31,13 @@ name](#the-active-target). *Program* = *Instruction* ... A configuration source file is a file whose name ends in `.pr` that contains zero or more -Preserves [text-syntax](https://preserves.dev/preserves.html#textual-syntax) +Preserves [text-syntax](https://preserves.dev/preserves-text.html) values, which are together interpreted as a sequence of *Instruction*s. **Comments.** [Preserves comments](https://preserves.dev/conventions.html#comments) are ignored. One unfortunate wart is that because Preserves comments are really -[annotations](https://preserves.dev/preserves.html#annotations), they are +[annotations](https://preserves.dev/preserves-text.html#annotations), they are required by the Preserves data model to be attached to some other value. Syntactically, this manifests as the need for *some non-comment following every comment*. In scripts written to date, often an empty *SequencingInstruction* serves to anchor comments at the end of a file: @@ -182,7 +182,7 @@ facet terminates, *Instruction* is run.     `dataspace` |     `timestamp` |     `facet` | - `stringify` *ConvenienceExpr* | +    `stringify` *ConvenienceExpr* |     *ValueExpr* Values can be destructured and new variables introduced into the environment with `let`, which @@ -248,30 +248,47 @@ special syntax for *[attenuated](../glossary.md#attenuation) entity references*, ## Attenuation Expressions -*AttenuationExpr* = `<* $`*var*` [`*Rewrite* ...`]>` +*AttenuationExpr* = `<* $`*var*` [`*Caveat* ...`]>` + +*Caveat* = +    `` | +    `` | +    *Rewrite* *Rewrite* = -    `` | +    `` |     `` An attenuation expression looks up *var* in the environment, asserts that it is an entity reference *orig*, and returns a new entity reference *ref*, like *orig* but -[attenuated](../glossary.md#attenuation) with zero or more *Rewrite*s. The result of evaluation +[attenuated](../glossary.md#attenuation) with zero or more *Caveat*s. The result of evaluation is *ref*, the new attenuated entity reference. -When an assertion is published or a message body arrives at *ref*, the sequence of *Rewrite*s -is executed left-to-right. If a *Rewrite* succeeds, the value if produces is forwarded on to -*orig*. If all *Rewrite*s fail, the assertion or message is silently ignored. +When an assertion is published or a message arrives at *ref*, the sequence of *Caveats*s is +executed **right-to-left**, transforming and possibly discarding the asserted value or message +body. If all *Caveat*s succeed, the final transformed value is forwarded on to *orig*. If any +*Caveat* fails, the assertion or message is silently ignored. -A `rewrite` *Rewrite* matches values with *PatternExpr*. If the match fails, the next *Rewrite* -is tried; if it succeeds, the resulting bindings are used along with the current environment to -evaluate *TemplateExpr*, and the resulting value is forwarded on to *orig*. +A *Caveat* can be one of three possibilities: -A `filter` *Rewrite* is the same as ` $`*v*`>`, for some fresh -*v*. + - An `or` of multiple alternative *Rewrite*s. The first *Rewrite* to accept (and possibly + transform) the input value causes the whole `or` *Caveat* to succeed. If all the *Rewrite*s + in the `or` fail, the `or` itself fails. Supplying a *Caveat* that is an `or` containing + zero *Rewrite*s will reject *all* assertions and messages. -Supplying zero *Rewrite*s will cause the new entity to reject *all* assertions and messages -sent to it. + - A `reject`, which allows all values through unchanged except those matching *PatternExpr*. + + - A simple *Rewrite*. + +A *Rewrite* can be one of two possibilities: + + - A `rewrite`, which matches input values with *PatternExpr*. If the match fails, the + *Rewrite* fails. If it succeeds, the resulting bindings are used along with the current + environment to evaluate *TemplateExpr*, and the *Rewrite* succeeds, yielding the resulting + value. + + - An `accept`, which is the same as ` $`*v*`>` for some fresh + *v*. ## Pattern Expressions @@ -348,10 +365,10 @@ binding the resulting capability to `$sys`. Any `require-service` record publish rewritten into a `require-core-service` record; other assertions are forwarded unchanged. ```preserves -let ?sys = <* $config [ +let ?sys = <* $config [ > - -]> + +]>]> ``` Then, `$sys` is used to build the initial environment for a [configuration diff --git a/src/operation/synit-config.md b/src/operation/synit-config.md index 19479a2..7d752cd 100644 --- a/src/operation/synit-config.md +++ b/src/operation/synit-config.md @@ -124,10 +124,10 @@ expression](./scripting.md#attenuation-expressions) rewrites `require-service` a `require-core-service` assertions: ```preserves -let ?sys = <* $config [ +let ?sys = <* $config [ > - -]> + +]>]> . +Caveat = Rewrite / Alts / Reject / @unknown any . +Rewrite = . +Reject = . Alts = . ``` -A `Caveat` can be either a single `Rewrite` or a sequence of alternative possible rewrites, -tried in left-to-right order until one of them accepts the input or there are none left to try. -(A single `Rewrite` *R* is equivalent to ``.) +A `Caveat` can be: -A `Rewrite` applies its `Pattern` to the input to the `Caveat`. If it matches, the bindings + - a single `Rewrite`[^single-rewrite-meaning], or a sequence of alternative possible rewrites + `Alts`, to be tried in left-to-right order until one of them accepts the input or there are + none left to try; + + - a `Reject`, which passes all inputs unmodified except those matching the contained pattern; or + + - an `unknown` caveat, which rejects all inputs. + +Each `Rewrite` applies its `Pattern` to its input. If the `Pattern` matches, the bindings captured by the pattern are gathered together and used in instantiation of the `Rewrite`'s `Template`, yielding the output from the `Caveat`. If the pattern does not match, the `Rewrite` has rejected the input, and other `alternatives` are tried until none remain, at which point @@ -518,6 +524,10 @@ The value is recursively traversed. As the relay comes across each embedded `Wir - In each case, the `WireSymbol` associated with the OID has its reference count incremented (if an `Assert` is being processed). +In addition, for `Assert` events, the `WireSymbol` (necessarily in the export membrane) +associated with the OID to which the incoming `Assert` is targetted has its reference count +incremented. + ### Rewriting embedded references for transmission When transmitting a `Value` *v* in an `Assert` or `Message` event, embedded references in *v* @@ -686,13 +696,13 @@ care should be taken in the case of non-cryptographic transport protocols like p To use such a transport for this protocol, establish a connection and begin transmitting [`Packet`s](#packet-definitions) encoded as Preserves values using either the Preserves [text -syntax](https://preserves.dev/preserves.html#textual-syntax) or the Preserves -[binary syntax](https://preserves.dev/preserves.html#compact-binary-syntax). +syntax](https://preserves.dev/preserves-text.html) or the Preserves +[machine-oriented syntax](https://preserves.dev/preserves-binary.html). The session starts with the first packet and ends with transport disconnection. If either peer in a connection detects a syntax error, it MUST disconnect the transport. A responding server MUST support the binary syntax, and MAY also support the text syntax. It can autodetect the syntax variant by following [the rules in the -specification](https://preserves.dev/preserves.html#appendix-autodetection-of-textual-or-binary-syntax): +specification](https://preserves.dev/preserves-binary.html#appendix-autodetection-of-textual-or-binary-syntax): the first byte of a valid binary-syntax Preserves document is guaranteed not to be interpretable as the start of a valid UTF-8 sequence. @@ -771,8 +781,9 @@ version 1 . Attenuation = [Caveat ...]. -Caveat = Rewrite / Alts . -Rewrite = . +Caveat = Rewrite / Alts / Reject / @unknown any . +Rewrite = . +Reject = . Alts = . Oid = int . @@ -793,7 +804,7 @@ PCompound = / @dict . Template = TAttenuate / TRef / Lit / TCompound . -TAttenuate = . +TAttenuate = . TRef = . TCompound = / @rec @@ -806,8 +817,8 @@ TCompound = ### Attenuation ```python -def attenuate(attenuation, value): - for caveat in reversed(attenuation): +def attenuate(caveats, value): + for caveat in reversed(caveats): value = applyCaveat(caveat, value) if value is None: return None @@ -822,6 +833,13 @@ def applyCaveat(caveat, value): return None if caveat is 'Rewrite' variant: return tryRewrite(caveat, value) + if caveat is 'Reject' variant: + if applyPattern(caveat.pattern, value) is None: + return value + else: + return None + if caveat is 'unknown' variant: + return None def tryRewrite(rewrite, value): bindings = applyPattern(rewrite.pattern, value) @@ -947,10 +965,6 @@ def instantiate(template, bindings): mechanism. Future versions of this specification may opt to include some of this generality. -[^zero-or-more]: TODO: It might be better to have a `Caveat` yield *zero or more* values? That - way they can act as filters. I've sometimes wanted the multiple-value case, though I've so - far been able to work around its lack. TODO: Perhaps it would also make sense to have a - `Caveat` map an *event* to zero or more *events*, rather than to values? Tricky corners - there include ensuring that carried authority isn't misused; macaroons are a very elegant - solution to this problem, of course, so maybe the macaroon design idea could be adapted to - this. For now, `Value`→`Option` is probably OK. +[^affine-caveats]: `Caveat`s are thus *affine*. + +[^single-rewrite-meaning]: A single `Rewrite` *R* is equivalent to ``. diff --git a/src/protocols/syndicate/gatekeeper.md b/src/protocols/syndicate/gatekeeper.md index 3663004..9fafddb 100644 --- a/src/protocols/syndicate/gatekeeper.md +++ b/src/protocols/syndicate/gatekeeper.md @@ -25,16 +25,28 @@ an overview of `SturdyRef`s, see the [guide to the built-in gatekeeper entity](../../operation/builtin/gatekeeper.md#sturdyrefs). ``` -SturdyRef = . +SturdyRef = . ``` -For detail of the interpretation of `Attenuation`s, `Caveat`s, `Pattern`s, and `Template`s, see -the [Syndicate protocol specification](../../protocol.md#capabilities-on-the-wire). +The `sig` in a `SturdyRef` is an iterated keyed-HMAC construction, starting from an HMAC of the +ref's *secret key* and its `oid`, following [macaroons][]. The specific function chosen is +[HMAC](https://www.rfc-editor.org/rfc/rfc2104) using +[BLAKE2s-256](https://www.rfc-editor.org/rfc/rfc7693), truncating the output to the first 16 +bytes. Let + + - *f(k,d)* be `HMAC-BLAKE2s-256`*(k,d)*[0..16), + - *e(v)* yield the canonical machine-oriented serialization of some preserves value *v*, and + - *k* be the original secret key for the ref. + +In a valid `SturdyRef`, then, the `sig` will be *f(...f(...f(f(k,e(*`oid`*)),...),e(*`Caveat`*)),...)*. + +For detail of the interpretation of `Caveat`s, `Pattern`s, and `Template`s, see the [Syndicate +protocol specification](../../protocol.md#capabilities-on-the-wire). ``` -Attenuation = [Caveat ...]. -Caveat = Rewrite / Alts . -Rewrite = . +Caveat = Rewrite / Alts / Reject / @unknown any . +Rewrite = . +Reject = . Alts = . Lit = . @@ -52,10 +64,12 @@ PCompound = / @dict . Template = TAttenuate / TRef / Lit / TCompound . -TAttenuate = . +TAttenuate = . TRef = . TCompound = / @rec / @arr / @dict . ``` + +[Macaroons]: ../../glossary.md#macaroon