From 5a89f3061305fbed2d9748cae414113282fca634 Mon Sep 17 00:00:00 2001 From: Tony Garnock-Jones Date: Mon, 7 Mar 2022 17:57:02 +0100 Subject: [PATCH] More manual text --- src/SUMMARY.md | 3 +- src/operation/builtin/config-watcher.md | 6 ++ src/operation/builtin/daemon.md | 3 + src/operation/builtin/gatekeeper.md | 72 +++++++++++++++ src/operation/builtin/index.md | 66 ++++++++++++- src/operation/builtin/logging.md | 1 + src/operation/builtin/relay-listener.md | 12 ++- src/operation/service.md | 117 ++++++++++++++++++++++++ 8 files changed, 277 insertions(+), 3 deletions(-) create mode 100644 src/operation/builtin/logging.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 0e3e635..f2c14df 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -16,7 +16,8 @@ - [Services and service dependencies](./operation/service.md) - [Built-in services and service classes](./operation/builtin/index.md) - [Gatekeeper](./operation/builtin/gatekeeper.md) - - [TCP/IP and Unix-socket Transports](./operation/builtin/relay-listener.md) + - [Logging](./operation/builtin/logging.md) + - [TCP/IP, WebSocket and Unix-socket Transports](./operation/builtin/relay-listener.md) - [Configuration watcher](./operation/builtin/config-watcher.md) - [Daemons and external programs](./operation/builtin/daemon.md) - [Configuration files and directories]() diff --git a/src/operation/builtin/config-watcher.md b/src/operation/builtin/config-watcher.md index 7484ce4..af39700 100644 --- a/src/operation/builtin/config-watcher.md +++ b/src/operation/builtin/config-watcher.md @@ -1 +1,7 @@ # Configuration watcher + + - Any number of [**configuration watchers**](./config-watcher.md) may be created to monitor + directories for changes to files written using the [server scripting + language](../scripting.md). (These can also be started from the `syndicate-server` + command-line with `-c` options.) + diff --git a/src/operation/builtin/daemon.md b/src/operation/builtin/daemon.md index 4cb35e0..3f4d07c 100644 --- a/src/operation/builtin/daemon.md +++ b/src/operation/builtin/daemon.md @@ -1 +1,4 @@ # Process supervision and management + + - Finally, [**external programs**](./daemon.md) can be started, either as long-lived "daemon" + services or as one-off scripts. diff --git a/src/operation/builtin/gatekeeper.md b/src/operation/builtin/gatekeeper.md index 6d79d3f..5f2b4db 100644 --- a/src/operation/builtin/gatekeeper.md +++ b/src/operation/builtin/gatekeeper.md @@ -1 +1,73 @@ # Gatekeeper + + - Relevant schema: + - Gatekeepers: [[syndicate-protocol]/schemas/gatekeeper.prs](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/branch/main/schemas/gatekeeper.prs) + - Sturdyrefs: [[syndicate-protocol]/schemas/sturdy.prs](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/branch/main/schemas/sturdy.prs) + +When `syndicate-server` starts, it creates a *gatekeeper service entity*, which accepts +`resolve` assertions requesting conversion of a long-lived "sturdyref" to a [live +reference](../../glossary.md#reference). The gatekeeper is the [default +object](../../glossary.md#initial-ref), available as [OID](../../glossary.md#oid) 0 to peers at +the other end of [relay listener](./relay-listener.md) connections. + +```preserves-schema +Resolve = . +``` + +When a request to resolve a given `sturdyref` appears, the gatekeeper entity queries a +dataspace (by default, the server's top-level `$config` dataspace) for `bind` assertions: + +```preserves-schema +Bind = . +``` + +Each `bind` assertion matching the requested `sturdyref` is checked against the credentials +provided in the sturdyref, and if the checks pass, the `target` entity from the `bind` is +asserted to the `observer` in the `resolve`. + +## Sturdyrefs + +A "sturdyref" is a long-lived certificate including a cryptographic signature that can be +upgraded by a gatekeeper entity to a live reference to the entity named in the sturdyref. The +current sturdyref implementation is based on the design of +[Macaroons][]. + +The following definitions are taken from the +[sturdy.prs](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/branch/main/schemas/sturdy.prs) +schema. + +```preserves-schema +SturdyRef = . +``` + +Within a `ref` record, the `oid` field is a free-form value that the targeted service chooses +to name itself. The `sig` is an iterated keyed-HMAC construction, just as in [macaroons][]. +First, the service's secret key is used to key an HMAC of the `oid`. Then, the result is used +to key an HMAC of the first `Attenuation` in `caveatChain`. Each `Attenuation`'s HMAC becomes +the key for the next in the `caveatChain`. The final result is equal to the `sig` field in a +valid sturdyref. + +### Attenuation of authority + +When it comes to publishing assertions or sending messages to the entity denoted by a +sturdyref, the `caveatChain` is used to [attenuate](../../glossary.md#attenuation) the +authority denoted by the sturdyref by filtering and/or rewriting assertion and message bodies. +The `caveatChain` is run *right to left*, with newer rewrites-and-filters at the right-hand end +of the chain and older ones at the left-hand end. Of course, an empty `caveatChain` is an +unattenuated reference. + +```preserves-schema +Attenuation = [Caveat ...]. +``` + +Each individual `Attenuation` in a `caveatChain` is a sequence of `Caveat`s. The term "caveat" +is shamelessly taken from [macaroons][], though our caveats presently embody only what in the +Macaroons paper are called "first-party caveats" over assertion structure; future versions of +the server may add "third-party caveats" and other, richer, predicates over assertions. + +Each `Attenuation`'s `Caveat`s are run in *right to left* order. + +For details of the structure and interpretation of `Caveat`s, see [the relevant section of the +Syndicate network protocol specification](../../protocol.md#attenuation-of-authority). + +[Macaroons]: ../../glossary.md#macaroon diff --git a/src/operation/builtin/index.md b/src/operation/builtin/index.md index 8284d21..ae79c9b 100644 --- a/src/operation/builtin/index.md +++ b/src/operation/builtin/index.md @@ -1,3 +1,67 @@ # Built-in services and service classes -The `syndicate-server` program +The `syndicate-server` program includes built-in knowledge about a handful of useful services, +including a means of loading external programs and integrating them into the running system. + + - Every server program starts a [**gatekeeper**](./gatekeeper.md) service, which is able to + manage conversion between [live references](../../glossary.md#reference) and so-called + "sturdy refs", long-lived capabilities for access to resources managed by the server. + + - A simple [**logging**](./logging.md) actor copies log messages from the system dataspace to + the server's standard error file descriptor. + + - Any number of [**TCP/IP, WebSocket, and Unix socket transports**](./relay-listener.md) may + be configured to allow external access to the gatekeeper and its registered services. (These + can also be started from the `syndicate-server` command-line with `-p` and `-s` options.) + + - Any number of [**configuration watchers**](./config-watcher.md) may be created to monitor + directories for changes to files written using the [server scripting + language](../scripting.md). (These can also be started from the `syndicate-server` + command-line with `-c` options.) + + - Finally, [**external programs**](./daemon.md) can be started, either as long-lived "daemon" + services or as one-off scripts. + +## Resources available at startup + +The `syndicate-server` program uses the Rust +[`tracing`](https://docs.rs/tracing/latest/tracing/) crate, which means different levels of +internal logging verbosity are available via the `RUST_LOG` environment variable. [See here for +more on `RUST_LOG`](https://docs.rs/env_logger/latest/env_logger/#enabling-logging). + +If tracing of [Syndicated Actor Model actions](../../glossary.md#action) is enabled with the +`-t` flag, it is configured prior to the start of the main server actor. + +As the main actor starts up, it + + - creates a fresh dataspace, known as the `$config` dataspace, intended to contain + top-level/global configuration for the server instance; + + - creates a fresh dataspace, known as `$log`, for assertions and messages related to service + logging within the server instance; + + - creates the `$gatekeeper` actor implementing the [gatekeeper](./gatekeeper.md) service, + attaching it to the `$config` dataspace; + + - exposes `$config`, `$log` and `$gatekeeper` as the variables available to configuration + scripts loaded by [config-watchers](./config-watcher.md) started with the `-c` flag (N.B. + the `$config` dataspace is thus the default target for assertions in config files); + + - creates service factories monitoring various [service assertions](../service.md) in the + `$config` dataspace; + + - processes `-p` command-line options, each of which creates a [TCP/IP relay + listener](./relay-listener.md#tcpip); + + - processes `-s` command-line options, each of which creates a [Unix socket relay + listener](./relay-listener.md#unix-sockets); + + - processes `-c` command-line options, each of which creates a + [config-watcher](./config-watcher.md) monitoring a file-system directory; and finally + + - creates the [logging actor](./logging.md), listening to certain events on the `$log` + dataspace. + +Once these tasks have been completed, it quiesces, leaving the rest of the operation of the +system up to other actors (relay-listeners, configuration scripts, and other configured +services). diff --git a/src/operation/builtin/logging.md b/src/operation/builtin/logging.md new file mode 100644 index 0000000..b921bbe --- /dev/null +++ b/src/operation/builtin/logging.md @@ -0,0 +1 @@ +# Logging diff --git a/src/operation/builtin/relay-listener.md b/src/operation/builtin/relay-listener.md index 5dbb54b..15d0923 100644 --- a/src/operation/builtin/relay-listener.md +++ b/src/operation/builtin/relay-listener.md @@ -1 +1,11 @@ -# TCP/IP and Unix-socket Transports +# TCP/IP, WebSocket and Unix-socket Transports + + - Any number of [**TCP/IP, WebSocket, and Unix socket transports**](./relay-listener.md) may + be configured to allow external access to the gatekeeper and its registered services. (These + can also be started from the `syndicate-server` command-line with `-p` and `-s` options.) + +## TCP/IP + +## WebSockets + +## Unix sockets diff --git a/src/operation/service.md b/src/operation/service.md index 68d05d2..c042495 100644 --- a/src/operation/service.md +++ b/src/operation/service.md @@ -1 +1,118 @@ # Services and service dependencies + + - Relevant schema source: + [[syndicate-protocols]/schemas/service.prs][service.prs] + +[service.prs]: https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/branch/main/schemas/service.prs + +Assertions in the main `$config` dataspace are the means Synit uses to declare services and +service dependencies. + +Service are started "gracefully", taking their dependencies into consideration, using +`require-service` assertions; upon appearance of `require-service`, and after dependencies are +satisfied, a `run-service` assertion is automatically made. Services can also be +"force-started" using `run-service` assertions directly. Once all `run-service` assertions for +a service have been withdrawn, services shut themselves down. + +## Example: Docker daemon + +As a concrete example, take the file +[`/etc/syndicate/services/docker.pr`](https://git.syndicate-lang.org/synit/synit/src/branch/main/packaging/packages/synit-config/files/etc/syndicate/services/docker.pr), +which both defines and invokes a service for running the Docker daemon: + +```preserves +> + up>> +/var/log/docker.log"> +``` + +This is an example of the [scripting language](./scripting.md) in action, albeit a simple one +without use of variables or any reactive constructs. + + - The `require-service` assertion instructs `syndicate-server` to solve the dependencies for + the service named `` and to start the service running. + + - The `depends-on` assertion specifies that the Docker daemon requires the `network` milestone + (configured primarily in + [network.pr](https://git.syndicate-lang.org/synit/synit/src/branch/main/packaging/packages/synit-config/files/etc/syndicate/services/network.pr)) + to have been reached. + + - The `daemon` assertion is interpreted by the [built-in external service + class](./builtin/daemon.md), and specifies how to configure and run the service once its + dependencies are ready. + +## Details + +A few different kinds of assertions, all declared in [the `service.prs` schema][service.prs], +form the heart of the system. + +### Assert that a service and its dependencies should be started + +```preserves-schema +RequireService = . +``` + +Asserts that a service should begin (and stay) running after waiting for its dependencies and +considering reverse-dependencies, blocks, and so on. + +### Assert that a service should start *right now* + +```preserves-schema +RunService = . +``` + +Asserts that a service should begin (and stay) running RIGHT NOW, without considering its +dependencies. + +The built-in handler for `require-service` assertions will assert `run-service` automatically +once all dependencies have been satisfied. + +### Declare a dependency among services + +```preserves-schema +ServiceDependency = . +``` + +Asserts that, when `depender` is `require-service`d, it should not be started until `dependee` +has been asserted, and also that `dependee`'s `serviceName` should be `require-service`d. + +### Convey the current state of a service + +```preserves-schema +ServiceState = . +State = =started / =ready / =failed / =complete / @userDefined any . +``` + +Asserts one or more current states of service `serviceName`. The overall state of the service +is the union of asserted `state`s. + +A few built-in states are defined: + + - `started` - the service has begun its startup routine, and may or may not be ready to take + requests from other parties. + + - `started` + `ready` - the service has started and is also ready to take requests from other + parties. Note that the `ready` state is special in that it is asserted *in addition* to + `started`. + + - `failed` - the service has failed. + + - `complete` - the service has completed execution. + +In addition, any user-defined value is acceptable as a `State`. + +### Make an entity representing a service instance available + +```preserves-schema +ServiceObject = . +``` + +A running service publishes zero or more of these. The details of the object vary by service. + +### Request a service restart + +```preserves-schema +RestartService = . +``` + +This is a *message*, not an assertion. It should be sent in order to request a service restart.