syndicate_utils/README.md

# Syndicate utils

## Syndesizer

A Syndicate multitool that includes a number of different actors that become active via configuration.

Think of it as a Busybox for Syndicate, if Busybox was created before POSIX.

Whether you use a single instance for many protocols or many specialized instances is up to you.

### Cache

Observes patterns and reässert the values captured for a given lifetime. Takes the argument `<cache { dataspace: #!any lifetime: float }>`. The lifetime of a cache counts down from moment a value is asserted.

Example configuration:
```
? <nixspace ?nixspace> [
  ; Require the nix_actor during observations.
  ?nixspace> ? <Observe <rec eval _> _> [
    $config <require-service <daemon nix_actor>> ]
  ?nixspace> ? <Observe <rec realise _> _> [
    $config <require-service <daemon nix_actor>> ]

  ; Cache anything captured by observers in the $nixspace for an hour.
  ; The nix_actor is not required during caching.
  $config <require-service <daemon syndesizer>>
  $config ? <service-object <daemon syndesizer> ?cap> [
    $cap <cache { dataspace: $nixspace lifetime: 3600.0 }> ]
]
```
### File System Usage

Summarize the size of file-system directory. Equivalent to `du -s -b`.
Query the size of a directory in bytes by observing `<file-system-usage "/SOME/PATH" ?size>`.

```
# Configuration example

? <exposed-dataspace ?ds> [

  <require-service <daemon syndesizer>>
  ? <service-object <daemon syndesizer> ?cap> [
    $cap <file-system-usage { dataspace: $ds }>
  ]

]
```

### JSON Socket Translator

Communicate with sockets that send and receive lines of JSON using `<send …>` and `<recv …>` messages.

Do not send messages into the dataspace configure with `<json-socket-translator …>` until `<connected @socketPath string>` is asserted.

```
# MPV configuration example
<require-service <daemon mpv-server>>

<daemon mpv-server {
  argv: [
    "/run/current-system/sw/bin/mpv"
    "--really-quiet"
    "--idle=yes"
    "--no-audio-display"
    "--input-ipc-server=/run/user/1000/mpv.sock"
    "--volume=75"
  ]
  protocol: none
}>

let ?mpvSpace = dataspace

? <service-state <daemon mpv-server> ready> [
  <require-service <daemon syndesizer>>
  ? <service-object <daemon syndesizer> ?cap> [
    $cap <json-socket-translator {
      dataspace: $mpvSpace
      socket: "/run/user/1000/mpv.sock"
    }>
  ]
]

$mpvSpace [

  # announce the dataspace when the translator is connected
  ? <connected $socketPath> [
    $config <mpv $mpvSpace>
    $config <bind <ref { oid: "mpv" key: #x"" }> $mpvSpace #f>
  ]

  # translate <play-file …> to an MPV command
  ?? <play-file ?file> [
    ! <send { "command": ["loadfile" $file "append-play"] }>
  ]

  # clear the playlist on idle so it doesn't grow indefinitely
  ?? <recv {"event": "idle"}> [
    ! <send { "command": ["playlist-clear"] }>
  ]
]
```

### JSON Stdio Translator

Executes a command, parses its JSON output, converts to record `<recv @jsonData any>`, and publishes and messages it to a dataspace.

```
# Configuration example
<require-service <daemon syndesizer>>

let ?ds = dataspace
<bind <ref {oid: "syndicate" key: #x""}> $ds #f>

? <service-object <daemon syndesizer> ?cap> [
  $cap <json-stdio-translator {
    argv: [
      "yt-dlp"
      "--dump-json"
      "https://youtu.be/RR9GkEXDvog"
    ]
    dataspace: $ds
  }>
]
```

### PostgreSQL

Readonly access to PostgreSQL databases. Asserts rows as records in response to SQL query assertions. Dynamic updates are not implemented.

Can be disabled by passing `--define:withPostgre=no` to the Nim compiler.

```
# Configuration example
<require-service <daemon syndesizer>>

let ?sqlspace = dataspace

? <service-object <daemon syndesizer> ?cap> [
  $cap <postgre {
     dataspace: $sqlspace
     connection: [
       ["host" "example.com"]
       ["dbname" "foobar"]
       ["user" "hackme"]
     ]
   }>
]

let ?tuplespace = dataspace

$sqlspace <query "SELECT id, name FROM stuff" $tuplespace>

$tuplespace ? [?id ?name] [
  $log ! <log "-" { row: <example-row $id $name> }>
]
```

### Pulse proxy

A proxy actor that passes assertions and messages to a configured capability but only asserts observations on a a periodic pulse.
This can be used to implement polling behavior.

```
# Example config
let ?ds = dataspace

<require-service <daemon syndesizer>>
? <service-object <daemon syndesizer> ?cap> [
  $cap <pulse {dataspace: $ds}>
]

$ds ? <pulse 3600.0 ?proxy> [
  $proxy ? <assertion-updated-hourly ?value> [
    $log ! <log "-" {assertion-updated-hourly: $value}>
  ]
]
```

### SQLite

Readonly access to SQLite databases. Asserts rows as records in response to SQL query assertions. Dynamic updates are not implemented.

Can be disabled by passing `--define:withSqlite=no` to the Nim compiler.

```
# Configuration example
<require-service <daemon syndesizer>>

let ?sqlspace = dataspace

? <service-object <daemon syndesizer> ?cap> [
  $cap <sqlite {
    dataspace: $sqlspace
    database: "/var/db/example.db"
  }>
]

let ?tuplespace = dataspace

$sqlspace <query "SELECT id, name FROM stuff" $tuplespace>

$tuplespace ? [?id ?name] [
  $log ! <log "-" { row: <example-row $id $name> }>
]
```

### Webooks

Listens for webhook requests and sends request data to a dataspace as messages.
Request data is formated according to the http schema [defined in syndicate-protocols](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/branch/main/schemas/http.prs), with the exception that messages bodies may be **bytes**, **string**, or **any** for the `content-type`s of `application/octet-stream`, `text/*`, and `application/json` respectively.

```
# Configuration example
<require-service <daemon syndesizer>>
? <service-object <daemon syndesizer> ?cap> [
  $cap <webhooks {
    listen: <tcp "0.0.0.0" 1048>
    endpoints: {

      # http://0.0.0.0:1048/my-endpoint
      ["my-endpoint"]: $target-dataspace

      # http://0.0.0.0:1048/some/multi-element/path
      ["some", "multi-element", "path"]: $target-dataspace

    }
  }>
]
```

### Websockets

connects to a websocket endpoint. During the lifetime of the connection a `<connected $URL>` assertion is made. Messages received from the server are sent to the dataspace wrapped in `<recv …>` records and messages observed as `<send …>` are sent to the server.

```
# Configuration example
<require-service <daemon syndesizer>>

let ?websocketspace = dataspace

? <service-object <daemon syndesizer> ?cap> [
  $cap <websocket {
    dataspace: $websocketspace
    url: "ws://127.0.0.1:5225/"
  }>
]

$websocketspace ? <connected $websocketUrl> [
  <bind <ref { oid: "websocket" key: #x"" }> $websocketspace #f>
]
```
### XML translator

Translates between Preserves and XML according to the [Conventions for Common Data Types](https://preserves.dev/conventions.html).

Examples:
- `<xml-translation "<foo a=\"1\"> <bar>hello world!</bar></foo>" <foo {"a": 1}<bar "hello world!">>>`
- `<xml-translation "" [#t #f]>`
- `<xml-translation "<<</>>" #f>`

```
# Configuration example
? <sharedspace ?ds> [
  $ds ? <Observe <rec xml-translation _> _> $config [
    $config <require-service <daemon syndesizer>>
    $config ? <service-object <daemon syndesizer> ?cap> [
      $cap <xml-translator { dataspace: $ds }>
    ]
  ]
]
```

### XSLT processor

Perform XML stylesheet transformations. For a given textual XSLT stylesheet and a textual XML document generate an abstract XML document in Preserves form. Inputs may be XML text or paths to XML files.

```
# Configuration example
let ?ds = dataspace
$ds [
  ? <xslt-transform "/stylesheet.xls" "/doc.xml" ?output> [
    ? <xml-translation ?text $output> [
      $log ! <log "-" { xslt-output: $text }>
    ]
  ]
]

<require-service <daemon syndesizer>>
? <service-object <daemon syndesizer> ?cap> $cap [
  <xml-translator { dataspace: $ds }>
  <xslt { dataspace: $ds }>
]
```

---

## mintsturdyref

A utility for minting [Sturdyrefs](https://synit.org/book/operation/builtin/gatekeeper.html#sturdyrefs).

## mount_actor

Actor for mounting filesystems on Linux.

Sample Syndicate server script:
```
# Assert a file-system we want to mount.
<mount "/dev/sda3" "/boot" "vfat">

# Transform mount assertions into mount status observations.
? <mount ?source ?target ?fs> [
  ? <mount $source $target $fs _> [ ]
]

# Assert mounting succeded.
? <mount _ ?target _ #t> [
  <service-state <mountpoint $target> ready>
]
# Assert mount failed.
? <mount _ ?target _ <failure _>> [
  <service-state <mountpoint $target> failed>
]

# Assert the details into the machine dataspace.
? <machine-dataspace ?machine> [
  $config ? <mount ?source ?target ?fs ?status> [
    $machine <mount $source $target $fs $status>
  ]
]

# Require the mount_actor daemon.
<require-service <daemon mount_actor>>
<daemon mount_actor {
  argv: ["/home/emery/src/bin/mount_actor"]
  protocol: application/syndicate
}>

# Pass the daemon the config dataspace.
? <service-object <daemon mount_actor> ?cap> [
  $cap { dataspace: $config }
]
```

## msg

A utility that sends messages to `$SYNDICATE_ROUTE`.


## net_mapper

Publishes ICMP packet round-trip-times. See [net_mapper.prs](./net_mapper.prs) for a protocol description. [Source](./src/net_mapper.nim).

Example script:
```
? <machine-dataspace ?machine> [
  $machine ? <rtt "10.0.33.136" ?min ?avg ?max> [
    $log ! <log "-" { ping: { min: $min avg: $avg max: $max } }>
  ]

  $config [
    <require-service <daemon net_mapper>>
    <daemon net_mapper {
      argv: ["/bin/net_mapper"]
      protocol: application/syndicate
    }>
    ? <service-object <daemon net_mapper> ?cap> [
      $cap { dataspace: $machine }
    ]
  ]
]
```

## preserve_process_environment

This utility serializes it's process environment to Preserves and prints it to stdout.
It can be used to feed the environment variables of a nested child of the Syndicate server back to the server. For example, to retreive the environmental variables that a desktop manager passed on to its children.