diff --git a/src/SUMMARY.md b/src/SUMMARY.md index d87a4ad..c6fb8a1 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -20,20 +20,7 @@ - [TCP/IP, WebSocket and Unix-socket Transports](./operation/builtin/relay-listener.md) - [Configuration watcher](./operation/builtin/config-watcher.md) - [Daemons and external programs](./operation/builtin/daemon.md) -- [Configuration files and directories]() - - [The boot layer]() - - [Logging]() - - ["Machine" dataspace]() - - ["exec" of one-off tasks]() - - [The core layer]() - - [The services layer]() - - [Networking core]() - - [Wifi & Mobile Data]() - - [Simple daemons]() - - [docker]() - - [ntpd]() - - [sshd]() - - [User settings]() +- [Configuration files and directories](./operation/synit-config.md) - [Defining services and service classes]() - [Scheduling tasks]() - [Timer tolerances]() @@ -78,11 +65,6 @@ - [Overview](./guide/index.md) - [Preserves](./guide/preserves.md) - [Working with schemas](./guide/working-with-schemas.md) -- [Language-neutral protocols]() -- [Rust]() -- [Python]() -- [Smalltalk]() -- [Bash (Shell)]() - [Capturing and rendering interaction traces](./guide/tracing.md) # Specifications and Theory diff --git a/src/operation/synit-config.md b/src/operation/synit-config.md new file mode 100644 index 0000000..aa53a70 --- /dev/null +++ b/src/operation/synit-config.md @@ -0,0 +1,493 @@ +# Configuration files and directories + + - On a running system: `/etc/syndicate/` + - Source repository: [[synit]/packaging/packages/synit-config/files/etc/syndicate](https://git.syndicate-lang.org/synit/synit/src/branch/main/packaging/packages/synit-config/files/etc/syndicate) + +The [root system bus](./system-bus.md#the-root-system-bus) is started with a `--config +/etc/syndicate/boot` command-line argument, which causes it to execute configuration scripts in +that directory. In turn, the `boot` directory contains instructions for loading configuration +from other locations on the filesystem. + +This section will examine the layout of the configuration scripts and directories. + +## The boot layer + +The files in +[/etc/syndicate/boot](https://git.syndicate-lang.org/synit/synit/src/branch/main/packaging/packages/synit-config/files/etc/syndicate/boot) +define the boot layer. + +### Console getty + +The first thing the boot layer does, in +[001-console-getty.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/boot/001-console-getty.pr), +is start a `getty` on `/dev/console`: + +```preserves +> + +``` + +### Ad-hoc execution of programs + +Next, in +[010-exec.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/boot/010-exec.pr), +it installs a handler that responds to messages requesting ad-hoc execution of programs: + +```preserves +?? [ + let ?id = timestamp + let ?facet = facet + let ?d = + > + + ? complete> [$facet ! stop] + ? failed> [$facet ! stop] +] +``` + +If the restart policy is not specified, it is defaulted to `on-error`: + +```preserves +?? ! +``` + +### "Milestone" pseudo-services + +Then, in +[010-milestone.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/boot/010-milestone.pr), +it defines how to respond to a request to run a "milestone" pseudo-service: + +```preserves +? > [ + started> + ready> +] +``` + +The definition is trivial—when requested, simply declare success—but useful in that a +"milestone" can be used as a proxy for a configuration state that other services can depend +upon. + +Concretely, milestones are used in two places at present: a `core` milestone declares that the +core layer of services is ready, and a `network` milestone declares that initial network +configuration is complete. + +### Synthesis of service state "up" + +The [definition of ServiceState](./service.md#convey-the-current-state-of-a-service) includes +`ready`, for long-running service programs, and `complete`, for successful exit (exit status 0) +of "one-shot" service programs. In +[010-service-state-up.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/boot/010-service-state-up.pr), +we declare an alias `up` that is asserted in either of these cases: + +```preserves +? +? +``` + +### Loading of "core" and "services" layers + +The final tasks of the boot layer are to load the "core" and "service" layers, respectively. + +Services declared in the "core" layer are automatically marked as dependencies of the +`` pseudo-service, and those declared in the "services" layer are automatically +marked as depending on ``. + +```ditaa + +------+ +-----+ +-------+ +----+ +----+ +------------+ +----+ + |docker| |modem| |network| |ntpd| |sshd| |userSettings| |wifi| + +---+--+ +--+--+ +---+---+ +--+-+ +--+-+ +------+-----+ +--+-+ + | | | | | | | + +-------+--------+--------+------+----------+----------+ + | depend on milestone core +services layer V + +----------------+ +- - - - - - - - - - - -| milestone core |- - - - - - - - - - - - - + +--------+-------+ + core layer | depended on by milestone core + +--------+--+-----------+ + | | | + V V V + +-----+ +--------+ +-----------------+ + |eudev| |hostname| |machine-dataspace| + +-----+ +--------+ +-----------------+ +``` + +#### The core layer loader + +For the core layer, in +[020-load-core-layer.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/boot/020-load-core-layer.pr), +a [configuration watcher](./builtin/config-watcher.md) is started, monitoring +`/etc/syndicate/core` for scripts defining services to place into the layer. Instead of passing +an unattenuated reference to `$config` to the configuration watcher, an [attenuation +expression](./scripting.md#attenuation-expressions) rewrites `require-service` assertions into +`require-core-service` assertions: + +```preserves +let ?sys = <* $config [ + > + +]> + + +``` + +Then, `require-core-service` is given meaning: + +```preserves +? [ + > + +] +``` + +#### The services layer loader + +The services layer is treated similarly in +[030-load-services.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/boot/030-load-services.pr), +except `require-basic-service` takes the place of `require-core-service`, and the configuration +watcher isn't started until `` is ready. Any `require-basic-service` assertions +are given meaning as follows: + +```preserves +? [ + up>> + +] +``` + +## The core layer: /etc/syndicate/core + +The files in +[/etc/syndicate/core](https://git.syndicate-lang.org/synit/synit/src/branch/main/packaging/packages/synit-config/files/etc/syndicate/core) +define the core layer. + +The +[configdirs.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/core/configdirs.pr) +script brings in scripts in `/run` and `/usr/local` analogues of the core config directory: + +```preserves +> +> +``` + +The +[eudev.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/core/eudev.pr) +script runs a `udevd` instance and, once it's ready, starts an initial scan: + +```preserves +> + + +> + up>> + /proc/sys/kernel/hotplug && + udevadm trigger --type=subsystems --action=add && + udevadm trigger --type=devices --action=add && + udevadm settle --timeout=30 +">> +``` + +The +[hostname.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/core/hostname.pr) +script simply sets the machine hostname: + +```preserves +> +> +``` + +Finally, the +[machine-dataspace.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/core/machine-dataspace.pr) +script declares a fresh, empty dataspace, and asserts a reference to it in a "well-known +location" for use by other services later: + +```preserves +let ?ds = dataspace + +``` + +## The services layer: /etc/syndicate/services + +The files in +[/etc/syndicate/services](https://git.syndicate-lang.org/synit/synit/src/branch/main/packaging/packages/synit-config/files/etc/syndicate/services) +define the services layer. + +The +[configdirs.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/configdirs.pr) +script brings in `/run` and `/usr/local` service definitions, analogous to the same file in the +core layer: + +```preserves +> +> +``` + +### Networking core + +The +[network.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/network.pr) +defines the `` pseudo-service and starts a number of ancillary services for +generically monitoring and configuring system network interfaces. + +First, `` is a small Python program, required by ``, using Netlink sockets to track changes to interfaces and interface state. It speaks +the [Syndicate network protocol](../protocol.md) on its standard input and output, and +publishes a [service object](./service.md#service-object) which expects a reference to the +[machine dataspace defined earlier](#machine-dataspace): + +```preserves +> + ready>> + +? [ + ? ?cap> [ + $cap { + machine: $machine + } + ] +] +``` + +The `interface-monitor` publishes assertions describing interface presence and state to the +machine dataspace. The network.pr script responds to these assertions by requesting +configuration of an interface once it reaches a certain state. First, all interfaces are +enabled when they appear and disabled when they disappear: + +```preserves + $machine ? [ + $config [ + ! + ?- ! + ] + ] +``` + +Next, a DHCP client is invoked for any "normal" (wired-ethernet-like) interface in "up" state +with a carrier: + +```preserves + $machine ? [ + $config > + ] + $machine ? [ + $config > + ] + + $config ? > [ + >> + ] + $config ? >> [ + ["udhcpc" "-i" $ifname "-fR" "-s" "/usr/lib/synit/udhcpc.script"]> + ] +``` + +We use a custom `udhcpc` script which modifies the default script to give mobile-data devices a +sensible routing metric. + +The final pieces of network.pr are static configuration of the loopback interface: + +```preserves +> +? > [ + ! + ?- ! +] +``` + +and conditional publication of a `default-route` record, allowing services to detect when the +internet is (nominally) available: + +```preserves + $machine ? [ + $config + ] +``` + +### Wifi & Mobile Data + +Building atop the networking core, +[wifi.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/wifi.pr) +and +[modem.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/modem.pr) +provide the necessary support for wireless LAN and mobile data interfaces, respectively. + +When `interface-monitor` detects presence of a wireless LAN interface, wifi.pr reacts by +starting `wpa_supplicant` for the interface along with a small Python program, `wifi-daemon`, +that acts as a client to `wpa_supplicant`, adding and removing networks and network +configuration according to `selected-wifi-network` assertions in the machine dataspace. + +```preserves + $machine ? [ + $config [ + >> + > + > up>> + >> + ] + ] + + $config ? >> [ + { + argv: "/usr/lib/synit/wifi-daemon" + protocol: application/syndicate + }> + ? > ?cap> [ + $cap { + machine: $machine + ifname: $ifname + } + ] + ] + + $config ? >> [ + [ + "wpa_supplicant" "-Dnl80211,wext" "-C/run/wpa_supplicant" "-i" $ifname + ]> + ] +``` + +The other tasks performed by wifi.pr are to request DHCP configuration for available wifi interfaces: + +```preserves + $machine ? [ + $config > + ] +``` + +and to relay `selected-wifi-network` records from [user settings](#user-settings) (described +below) into the machine dataspace, for `wifi-daemon` instances to pick up: + +```preserves + $config ? >> [ $machine += $s ] +``` + +Turning to modem.pr, which is currently hard-coded for Pinephone devices, we see two main +blocks of config. The simplest just starts the `eg25-manager` daemon for controlling the +Pinephone's Quectel modem, along with a simple monitoring script for restarting it if and when +`/dev/EG25.AT` disappears: + +```preserves + + up>> + +``` + +The remainder of modem.pr handles cellular data, configured via the +[qmicli](https://www.freedesktop.org/wiki/Software/libqmi/) program. + +```preserves +> + up>> +``` + +When the [user settings](#user-setting) `mobile-data-enabled` and `mobile-data-apn` are both +present, it responds to `qmi-wwan` service requests by invoking `qmi-wwan-manager`, a small +shell script, for each particular device and APN combination: + +```preserves +? > [ + ? > [ + ? > [ + >> + ] + ] +] +? >> [ + ["/usr/lib/synit/qmi-wwan-manager" $dev $apn]> +] +``` + +(Because qmicli is sometimes not well behaved, there is also [code in +modem.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/modem.pr#L23-L44) +for restarting it in certain circumstances when it gets into a state where it reports errors +but does not terminate.) + +### Simple daemons + +A few simple daemons are also started as part of the services layer. + +The +[docker.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/docker.pr) +script starts the docker daemon, but only once the network configuration is available: + +```preserves +> + up>> +/var/log/docker.log"> +``` + +The +[ntpd.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/ntpd.pr) +script starts an NTP daemon, but only when an IPv4 default route exists: + +```preserves +> + > + +``` + +Finally, the +[sshd.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/sshd.pr) +script starts the OpenSSH server daemon after ensuring both that the network is available and +that SSH host keys exist: + +```preserves +> + up>> + complete>> + +> +``` + +### User settings + +A special folder, `/etc/syndicate/user-settings`, acts as a persistent database of assertions +relating to user settings, including such things as wifi network credentials and preferences, +mobile data preferences, and so on. The +[userSettings.pr](https://git.syndicate-lang.org/synit/synit/src/commit/a43247fc96e906fbfdeb6e9a6e6d6e9a2f35f9e8/packaging/packages/synit-config/files/etc/syndicate/services/userSettings.pr) +script sets up the programs responsible for managing the folder. + +The contents of the folder itself are managed by a small Python program, +`user-settings-daemon`, which responds to requests arriving via the `$config` dataspace by +adding and removing files containing assertions in `/etc/syndicate/user-settings`. + +```preserves +let ?settingsDir = "/etc/syndicate/user-settings" +> + +? ?cap> [ + $cap { + config: $config + settingsDir: $settingsDir + } +] +``` + +Each such file is named after the SHA-1 digest of the [canonical +form](../guide/preserves.md#canonical-form) of the assertion it contains. For example, +`/etc/syndicate/user-settings/8814297f352be4ebbff19137770e619b2ebc5e91.pr` contains +``. + +The files in `/etc/syndicate/user-settings` are brought into the main config dataspace by way +of a rewriting configuration watcher: + +```preserves +let ?settings = <* $config [ > ]> +> +``` + +Every assertion from `/etc/syndicate/user-settings` is wrapped in a `` record +before being placed into the main `$config` dataspace.