syndicate-lang
/
preserves
mirror of https://gitlab.com/preserves/preserves.git
1
0
Fork 1
Code Releases Activity

Merge branch 'main' into newrust/main

This commit is contained in:
Tony Garnock-Jones 2023-03-24 19:19:55 +01:00
parent 4e51ef8d57 fe9d836942
commit 966bf7e35d
192 changed files with 45403 additions and 2578 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -38,7 +38,7 @@ automatic, perfect-fidelity conversion between syntaxes.
Implementations of the data model, plus the textual and/or binary transfer syntaxes:
- [Preserves for Nim](https://git.syndicate-lang.org/ehmry/preserves-nim)
- [Preserves for Python]({{page.projecttree}}/implementations/python/) ([`pip install preserves`](https://pypi.org/project/preserves/))
- [Preserves for Python]({{page.projecttree}}/implementations/python/) ([`pip install preserves`](https://pypi.org/project/preserves/); [documentation available online](python/latest/))
- [Preserves for Racket]({{page.projecttree}}/implementations/racket/preserves/) ([`raco pkg install preserves`](https://pkgs.racket-lang.org/package/preserves))
- [Preserves for Rust]({{page.projecttree}}/implementations/rust/) ([crates.io package](https://crates.io/crates/preserves))
- [Preserves for Squeak Smalltalk](https://squeaksource.com/Preserves.html) (`Installer ss project: 'Preserves'; install: 'Preserves'`)

6
_config.yml
View File

@ -1,4 +1,4 @@
exclude: [implementations]
exclude: [implementations, scratch]
markdown: kramdown
highlighter: rouge
@ -13,5 +13,5 @@ defaults:
layout: page
title: "Preserves"
version_date: "June 2022"
version: "0.6.3"
version_date: "March 2023"
version: "0.7.0"

4
_data/python-versions.json Normal file
View File

@ -0,0 +1,4 @@
[
{"version":"0.18.1","title":"0.18.1","aliases":["latest"]},
{"version":"0.18.0","title":"0.18.0","aliases":[]}
]

8
_includes/python-representation.md Normal file
View File

@ -0,0 +1,8 @@
Python's strings, byte strings, integers, booleans, and double-precision floats stand directly
for their Preserves counterparts. Wrapper objects for [Float][preserves.values.Float] and
[Symbol][preserves.values.Symbol] complete the suite of atomic types.
Python's lists and tuples correspond to Preserves `Sequence`s, and dicts and sets to
`Dictionary` and `Set` values, respectively. Preserves `Record`s are represented by
[Record][preserves.values.Record] objects. Finally, embedded values are represented by
[Embedded][preserves.values.Embedded] objects.

17
_includes/value-grammar.md Normal file
View File

@ -0,0 +1,17 @@
Value = Atom
| Compound
| Embedded
Atom = Boolean
| Float
| Double
| SignedInteger
| String
| ByteString
| Symbol
Compound = Record
| Sequence
| Set
| Dictionary

16
_includes/what-is-preserves-schema.md Normal file
View File

@ -0,0 +1,16 @@
A Preserves schema connects Preserves `Value`s to host-language data
structures. Each definition within a schema can be processed by a
compiler to produce
- a simple host-language *type definition*;
- a partial *parsing* function from `Value`s to instances of the
produced type; and
- a total *serialization* function from instances of the type to
`Value`s.
Every parsed `Value` retains enough information to always be able to
be serialized again, and every instance of a host-language data
structure contains, by construction, enough information to be
successfully serialized.

12
_includes/what-is-preserves.md Normal file
View File

@ -0,0 +1,12 @@
*Preserves* is a data model, with associated serialization formats.
It supports *records* with user-defined *labels*, embedded
*references*, and the usual suite of atomic and compound data types,
including *binary* data as a distinct type from text strings. Its
*annotations* allow separation of data from metadata such as comments,
trace information, and provenance information.
Preserves departs from many other data languages in defining how to
*compare* two values. Comparison is based on the data model, not on
syntax or on data structures of any particular implementation
language.

31
_layouts/page.html
View File

@ -4,5 +4,32 @@ extra_html_headers: >
<link rel="stylesheet" href="{{ site.baseurl }}/normalize.css">
<link rel="stylesheet" href="{{ site.baseurl }}/preserves.css">
---
<h1>{{ page.title }}</h1>
{{ content }}
<nav>
<div class="left">
<h1>
<a href="{{ site.baseurl }}/">
<span class="icon">
<img src="{{ site.baseurl }}/logo-64x64.png">
</span>
</a>
</h1>
</div>
<div class="middle">
<ul>
<li><a href="{{site.baseurl}}/preserves.html">Core</a>
<li><a href="{{site.baseurl}}/preserves-schema.html">Schema</a>
<li><a href="{{site.baseurl}}/preserves-path.html">Path</a>
</ul>
</div>
<div class="right">
<a href="https://gitlab.com/preserves/preserves">
<span class="icon">
<img src="{{ site.baseurl }}/gitlab-logo-500.svg" alt="Gitlab logo">
</span>
</a>
</div>
</nav>
<main>
<h1>{{ page.title }}</h1>
{{ content }}
</main>

6
_layouts/redirect.html
View File

@ -4,6 +4,8 @@ extra_html_headers: |
<link rel="canonical" href="{{ site.baseurl }}{{ page.redirect_target }}">
<meta http-equiv="Refresh" content="0; URL={{ site.baseurl }}{{ page.redirect_target }}">
---
<h1>Redirecting</h1>
<main>
<h1>Redirecting</h1>
<p>Redirecting you to <a href="{{ site.baseurl }}{{ page.redirect_target }}">{{ page.redirect_target }}</a>.</p>
<p>Redirecting you to <a href="{{ site.baseurl }}{{ page.redirect_target }}">{{ page.redirect_target }}</a>.</p>
</main>

4
_layouts/skeleton.html
View File

@ -3,6 +3,10 @@
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>{% unless page.no_site_title %}{{ site.title }}: {% endunless %}{{ page.title }}</title>
<link href="{{ site.baseurl }}/logo-64x64.png" rel="icon" />
<link href="{{ site.baseurl }}/logo-256x256.png" rel="shortcut icon" />
<meta name="author" content="Tony Garnock-Jones">
<meta name="viewport" content="width=device-width, initial-scale=1.0">{{
page.extra_html_headers | liquify }}{{

6
conventions.md
View File

@ -49,6 +49,12 @@ treat them specially.
and one which enforces validity (i.e. side-conditions) when reading,
writing, or constructing `Value`s.
## Metaconventions.
By and large `Capitalized` and `CamelCase` identifiers refer to *types* or *schema definition
names* describing families of `Value`s, while `kebab-case`, `lisp-style` identifiers are used
for concrete symbols appearing in e.g. `Record` labels.
## IOLists.
Inspired by Erlang's notions of

BIN
craiyon_111353.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 506 KiB

1
gitlab-logo-200.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 990 380"><defs><style>.cls-1{fill:#e24329;}.cls-2{fill:#fc6d26;}.cls-3{fill:#fca326;}.cls-4{fill:#fff;}</style></defs><g id="LOGO"><path class="cls-1" d="M302,174.37l-.21-.56-21.2-55.3a5.5,5.5,0,0,0-2.18-2.63,5.6,5.6,0,0,0-8.41,3.2l-14.31,43.81H197.74l-14.31-43.81a5.61,5.61,0,0,0-8.41-3.2,5.5,5.5,0,0,0-2.18,2.63l-21.19,55.31-.22.55a39.36,39.36,0,0,0,13.06,45.49l.08.06.18.14L197,244.23l16,12.09,9.72,7.35a6.57,6.57,0,0,0,7.92,0l9.72-7.35,16-12.09,32.48-24.31.09-.07A39.36,39.36,0,0,0,302,174.37Z"/><path class="cls-2" d="M302,174.37l-.21-.56a71.5,71.5,0,0,0-28.5,12.82l-46.55,35.2,29.64,22.4,32.48-24.31.09-.07A39.36,39.36,0,0,0,302,174.37Z"/><path class="cls-3" d="M197,244.23l16,12.09,9.72,7.35a6.57,6.57,0,0,0,7.92,0l9.72-7.35,16-12.09-29.64-22.4Z"/><path class="cls-2" d="M180.14,186.63a71.44,71.44,0,0,0-28.49-12.81l-.22.55a39.36,39.36,0,0,0,13.06,45.49l.08.06.18.14L197,244.23l29.66-22.4Z"/><path class="cls-4" d="M428.92,171.51H451.7c-3.8-24.22-24.77-41.09-52.06-41.09-32.29,0-56.52,23.74-56.52,63.5,0,39.05,23.14,63.27,57.18,63.27,30.55,0,52.42-19.65,52.42-51.46V190.91H402.65v17.47h28.44c-.36,17.6-12.11,28.74-30.67,28.74-20.66,0-34.82-15.48-34.82-43.44,0-27.78,14.4-43.2,34.34-43.2C414.82,150.48,425,158.43,428.92,171.51Z"/><path class="cls-4" d="M467.78,255.5h21.81V163H467.78Zm11-107.2c6.93,0,12.59-5.31,12.59-11.81s-5.66-11.87-12.59-11.87-12.65,5.3-12.65,11.87S471.75,148.3,478.74,148.3Z"/><path class="cls-4" d="M554.9,163H536.64V140.78H514.83V163H501.7v16.87h13.13v51.46c-.12,17.41,12.54,26,28.92,25.49a44.29,44.29,0,0,0,12.84-2.17l-3.68-17.06a26.57,26.57,0,0,1-6.38.85c-5.49,0-9.89-1.93-9.89-10.73V179.82H554.9Z"/><path class="cls-4" d="M571.78,255.5h76.7V236.76H594.14V132.1H571.78Z"/><path class="cls-4" d="M690.26,257.37c14.52,0,23.19-6.81,27.17-14.58h.72V255.5h21V193.56c0-24.46-19.94-31.81-37.6-31.81-19.46,0-34.4,8.67-39.22,25.54l20.37,2.9c2.16-6.33,8.31-11.75,19-11.75,10.13,0,15.67,5.18,15.67,14.28v.36c0,6.26-6.57,6.57-22.9,8.31-17.95,1.93-35.12,7.29-35.12,28.14C659.29,247.73,672.6,257.37,690.26,257.37Zm7.17-16c-9.1,0-15.61-4.16-15.61-12.17,0-8.38,7.29-11.87,17-13.26,5.73-.78,17.18-2.23,20-4.51v10.9C718.88,232.6,710.56,241.34,697.43,241.34Z"/><path class="cls-4" d="M755.21,255.5h21.45V240.92h1.26c3.44,6.75,10.61,16.21,26.52,16.21,21.81,0,38.14-17.3,38.14-47.78,0-30.85-16.81-47.6-38.2-47.6-16.33,0-23.14,9.82-26.46,16.51H777V132.1H755.21Zm21.39-46.27c0-18,7.71-29.59,21.75-29.59,14.52,0,22,12.35,22,29.59s-7.59,30-22,30C784.43,239.23,776.6,227.18,776.6,209.23Z"/></g></svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.5 KiB

1
gitlab-logo-500.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 380 380"><defs><style>.cls-1{fill:#e24329;}.cls-2{fill:#fc6d26;}.cls-3{fill:#fca326;}</style></defs><g id="LOGO"><path class="cls-1" d="M282.83,170.73l-.27-.69-26.14-68.22a6.81,6.81,0,0,0-2.69-3.24,7,7,0,0,0-8,.43,7,7,0,0,0-2.32,3.52l-17.65,54H154.29l-17.65-54A6.86,6.86,0,0,0,134.32,99a7,7,0,0,0-8-.43,6.87,6.87,0,0,0-2.69,3.24L97.44,170l-.26.69a48.54,48.54,0,0,0,16.1,56.1l.09.07.24.17,39.82,29.82,19.7,14.91,12,9.06a8.07,8.07,0,0,0,9.76,0l12-9.06,19.7-14.91,40.06-30,.1-.08A48.56,48.56,0,0,0,282.83,170.73Z"/><path class="cls-2" d="M282.83,170.73l-.27-.69a88.3,88.3,0,0,0-35.15,15.8L190,229.25c19.55,14.79,36.57,27.64,36.57,27.64l40.06-30,.1-.08A48.56,48.56,0,0,0,282.83,170.73Z"/><path class="cls-3" d="M153.43,256.89l19.7,14.91,12,9.06a8.07,8.07,0,0,0,9.76,0l12-9.06,19.7-14.91S209.55,244,190,229.25C170.45,244,153.43,256.89,153.43,256.89Z"/><path class="cls-2" d="M132.58,185.84A88.19,88.19,0,0,0,97.44,170l-.26.69a48.54,48.54,0,0,0,16.1,56.1l.09.07.24.17,39.82,29.82s17-12.85,36.57-27.64Z"/></g></svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.0 KiB

1
gitlab-logo-700.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 380 380"><defs><style>.cls-1{fill:#fff;}</style></defs><g id="LOGO"><path class="cls-1" d="M282.83,170.73l-.27-.69-26.14-68.22a6.81,6.81,0,0,0-2.69-3.24,7,7,0,0,0-8,.43,7,7,0,0,0-2.32,3.52l-17.65,54H154.29l-17.65-54A6.86,6.86,0,0,0,134.32,99a7,7,0,0,0-8-.43,6.87,6.87,0,0,0-2.69,3.24L97.44,170l-.26.69a48.54,48.54,0,0,0,16.1,56.1l.09.07.24.17,39.82,29.82,19.7,14.91,12,9.06a8.07,8.07,0,0,0,9.76,0l12-9.06,19.7-14.91,40.06-30,.1-.08A48.56,48.56,0,0,0,282.83,170.73Z"/></g></svg>
Side by Side

After

Width:  |  Height:  |  Size: 530 B

2
implementations/javascript/.dir-locals.el
View File

@ -15,7 +15,7 @@
"node_modules/")))
(lsp-dependency 'typescript-language-server
`(:system ,(concat node-modules
"typescript-language-server/lib/cli.js")))
"typescript-language-server/lib/cli.mjs")))
(lsp-dependency 'typescript
`(:system ,(concat node-modules
"typescript/lib/tsserver.js")))))

10
implementations/javascript/package.json
View File

@ -2,15 +2,15 @@
"name": "@preserves/root",
"private": true,
"devDependencies": {
"@rollup/plugin-terser": "^0.4",
"@types/jest": "^27.0",
"jest": "^27.4",
"lerna": "^4.0",
"rollup": "^2.60",
"rollup-plugin-terser": "^7.0",
"rollup": "^3.10",
"ts-jest": "^27.0",
"ts-node-dev": "^1.1",
"typescript": "^4.5",
"typescript-language-server": "^0.9.1"
"typescript": "^4.9",
"typescript-language-server": "^3.0"
},
"workspaces": [
"packages/*"
@ -18,7 +18,7 @@
"scripts": {
"prepare": "lerna exec yarn run prepare",
"clean": "lerna exec yarn run clean",
"veryclean": "yarn run veryclean:local && lerna exec yarn run veryclean",
"veryclean": "lerna exec yarn run veryclean && yarn run veryclean:local",
"veryclean:local": "rm -rf node_modules",
"build": "lerna exec yarn run prepare",
"test": "lerna exec yarn run test"

2
implementations/javascript/packages/core/package.json
View File

@ -1,6 +1,6 @@
{
"name": "@preserves/core",
"version": "0.20.4",
"version": "0.20.6",
"description": "Preserves data serialization format",
"homepage": "https://gitlab.com/preserves/preserves",
"license": "Apache-2.0",

2
implementations/javascript/packages/core/rollup.config.js → implementations/javascript/packages/core/rollup.config.mjs
View File

@ -1,4 +1,4 @@
import { terser } from 'rollup-plugin-terser';
import terser from '@rollup/plugin-terser';
const distfile = (insertion) => `dist/preserves${insertion}.js`;

4
implementations/javascript/packages/core/src/compound.ts
View File

@ -6,3 +6,7 @@ export function isCompound<T = GenericEmbedded>(x: Value<T>): x is Compound<T>
{
return (Array.isArray(x) || Set.isSet(x) || Dictionary.isDictionary(x));
}
export function isSequence<T = GenericEmbedded>(x: Value<T>): x is Array<Value<T>> {
return (Array.isArray(x) && !('label' in x));
}

4
implementations/javascript/packages/schema/package.json
View File

@ -1,6 +1,6 @@
{
"name": "@preserves/schema",
"version": "0.21.6",
"version": "0.21.10",
"description": "Schema support for Preserves data serialization format",
"homepage": "https://gitlab.com/preserves/preserves",
"license": "Apache-2.0",
@ -30,7 +30,7 @@
"preserves-schemac": "./bin/preserves-schemac.js"
},
"dependencies": {
"@preserves/core": "^0.20.4",
"@preserves/core": "^0.20.6",
"@types/glob": "^7.1",
"@types/minimatch": "^3.0",
"chalk": "^4.1",

2
implementations/javascript/packages/schema/rollup.config.js → implementations/javascript/packages/schema/rollup.config.mjs
View File

@ -1,4 +1,4 @@
import { terser } from 'rollup-plugin-terser';
import terser from '@rollup/plugin-terser';
const distfile = (insertion) => `dist/preserves-schema${insertion}.js`;

4
implementations/javascript/packages/schema/src/compiler/genconverter.ts
View File

@ -91,7 +91,7 @@ function converterForTuple(ctx: FunctionContext,
return knownArray
? loop(0)
: [seq(`if (_.Array.isArray(${src})`, lengthCheck, `) `, ctx.block(() => loop(0)))];
: [seq(`if (_.isSequence(${src})`, lengthCheck, `) `, ctx.block(() => loop(0)))];
}
function converterFor(
@ -157,7 +157,7 @@ export function converterForSimple(
return kKnownArray();
} else {
return [`${dest} = void 0`,
seq(`if (_.Array.isArray(${src})) `, ctx.block(kKnownArray))];
seq(`if (_.isSequence(${src})) `, ctx.block(kKnownArray))];
}
}
case 'setof':

4
implementations/javascript/packages/schema/src/compiler/genunconverter.ts
View File

@ -53,10 +53,10 @@ function unconverterFor(ctx: FunctionContext, p: M.Pattern, src: string): Item {
unconverterFor(ctx, M.Pattern.SimplePattern(p.pattern), 'v'),
`)`);
case 'setof':
return seq(`new _.Set<_embedded>`, parens(
return seq(`new _.Set<_embedded>`, parens(seq(
`_.Array.from(${src}.values()).map(v => `,
unconverterFor(ctx, M.Pattern.SimplePattern(p.pattern), 'v'),
`)`));
`)`)));
case 'dictof':
return seq(`new _.Dictionary<_embedded>`, parens(seq(
`_.Array.from(${src}.entries()).map(([k, v]) => `,

66
implementations/javascript/packages/schema/src/gen/schema.ts
View File

File diff suppressed because one or more lines are too long

2
implementations/javascript/packages/schema/src/reader.ts
View File

@ -254,7 +254,7 @@ function parsePattern(name: symbol, body0: Array<Input>): Pattern {
return ks(M.SimplePattern.seqof(walkSimple(item[0])));
} else if (Set.isSet<M.Input>(item)) {
if (item.size !== 1) complain();
const [vp] = item.entries();
const [vp] = item.values();
return ks(M.SimplePattern.setof(walkSimple(vp)));
} else if (Dictionary.isDictionary<M.InputEmbedded, Input>(item)
&& item.size === 2

4118
implementations/javascript/yarn.lock
View File

File diff suppressed because it is too large Load Diff

5
implementations/python/.envrc
View File

@ -2,7 +2,10 @@ if ! [ -d .venv ]
then
python -m venv .venv
. .venv/bin/activate
pip install -U coverage setuptools setuptools_scm wheel
pip install -U coverage setuptools setuptools_scm wheel \
mkdocs 'mkdocstrings[python]' mkdocs-material mkdocs-macros-plugin \
mkdocs-git-revision-date-localized-plugin
pip install -e .
else
. .venv/bin/activate
fi

1
implementations/python/.gitignore vendored
View File

@ -4,3 +4,4 @@ htmlcov/
build/
dist/
*.egg-info/
/.venv/

15
implementations/python/Makefile
View File

@ -1,6 +1,17 @@
PACKAGEVERSION="`python3 setup.py --version`"
all: test build-docs
test: update-test-data
python3 -m unittest discover -s tests
build-docs:
mkdocs build -d ../../python/$(PACKAGEVERSION)
./update-doc-versions.sh
serve-docs:
mkdocs serve
coverage: update-test-data
python3-coverage run --branch -m unittest discover -s tests
python3-coverage html
@ -9,7 +20,7 @@ update-test-data:
rsync ../../tests/samples.bin ../../tests/samples.pr tests
tag:
git tag python-preserves@`python3 setup.py --version`
git tag python-preserves@$(PACKAGEVERSION)
clean:
rm -rf htmlcov
@ -23,5 +34,5 @@ clean:
publish: clean build
twine upload dist/*
build:
build: build-docs
python3 setup.py sdist bdist_wheel

23
implementations/python/README.md Normal file
View File

@ -0,0 +1,23 @@
# Python Preserves
This package ([`preserves` on pypi.org](https://pypi.org/project/preserves/)) implements
[Preserves](https://preserves.dev/) for Python 3.x. It provides the core [semantics][] as well
as both the [human-readable text syntax](https://preserves.dev/preserves-text.html) (a superset
of JSON) and [machine-oriented binary format](https://preserves.dev/preserves-binary.html)
(including [canonicalization](https://preserves.dev/canonical-binary.html)) for Preserves. It
also implements [Preserves Schema](https://preserves.dev/preserves-schema.html) and [Preserves
Path](https://preserves.dev/preserves-path.html).
# Git repository
The project is [hosted on Gitlab](https://gitlab.com/preserves/preserves).
# Documentation
Documentation for the package is available at <https://preserves.dev/python/>.
# License
The package is licensed under the Apache License v2.0.
[semantics]: https://preserves.dev/preserves.html#semantics

3
implementations/python/docs/api.md Normal file
View File

@ -0,0 +1,3 @@
# The top-level preserves package
::: preserves

3
implementations/python/docs/binary.md Normal file
View File

@ -0,0 +1,3 @@
# Machine-oriented binary syntax
::: preserves.binary

3
implementations/python/docs/compare.md Normal file
View File

@ -0,0 +1,3 @@
# Comparing Values
::: preserves.compare

3
implementations/python/docs/error.md Normal file
View File

@ -0,0 +1,3 @@
# Codec errors
::: preserves.error

3
implementations/python/docs/fold.md Normal file
View File

@ -0,0 +1,3 @@
# Traversing values
::: preserves.fold

28
implementations/python/docs/index.md Normal file
View File

@ -0,0 +1,28 @@
# Overview
```shell
pip install preserves
```
This package ([`preserves` on pypi.org](https://pypi.org/project/preserves/)) implements
[Preserves](https://preserves.dev/) for Python 3.x. It provides the core [semantics][] as well
as both the [human-readable text syntax](text) (a superset of JSON) and [machine-oriented
binary format](binary) (including
[canonicalization](https://preserves.dev/canonical-binary.html)) for Preserves. It also
implements [Preserves Schema](schema) and [Preserves Path](path).
- Main package API: [preserves](api)
## What is Preserves?
{% include "what-is-preserves.md" %}
## Mapping between Preserves values and Python values
Preserves `Value`s are categorized in the following way:
{% include "value-grammar.md" %}
{% include "python-representation.md" %}
[semantics]: https://preserves.dev/preserves.html#semantics

3
implementations/python/docs/merge.md Normal file
View File

@ -0,0 +1,3 @@
# Merging values
::: preserves.merge

3
implementations/python/docs/path.md Normal file
View File

@ -0,0 +1,3 @@
# Preserves Path
::: preserves.path

7
implementations/python/docs/schema.md Normal file
View File

@ -0,0 +1,7 @@
# Preserves Schema
{% include "what-is-preserves-schema.md" %}
## Schema support in Python
::: preserves.schema

40
implementations/python/docs/syndicate-protocols-schema-bundle.bin Normal file
View File

@ -0,0 +1,40 @@
´³bundle·µ³tcp„´³schema·³version‘³ definitions·³TcpLocal´³rec´³lit³ tcp-local„´³tupleµ´³named³host´³atom³String„„´³named³port´³atom³ SignedInteger„„„„„³ TcpRemote´³rec´³lit³
tcp-remote„´³tupleµ´³named³host´³atom³String„„´³named³port´³atom³ SignedInteger„„„„„³ TcpPeerInfo´³rec´³lit³tcp-peer„´³tupleµ´³named³handle´³embedded³any„„´³named³local´³refµ„³TcpLocal„„´³named³remote´³refµ„³ TcpRemote„„„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„µ³http„´³schema·³version‘³ definitions·³Chunk´³orµµ±string´³atom³String„„µ±bytes´³atom³
ByteString„„„„³Headers´³dictof´³atom³Symbol„´³atom³String„„³MimeType´³atom³Symbol„³
QueryValue´³orµµ±string´³atom³String„„µ±file´³rec´³lit³file„´³tupleµ´³named³filename´³atom³String„„´³named³headers´³refµ„³Headers„„´³named³body´³atom³
ByteString„„„„„„„„³ HostPattern´³orµµ±host´³atom³String„„µ±any´³lit€„„„„³ HttpBinding´³rec´³lit³ http-bind„´³tupleµ´³named³host´³refµ„³ HostPattern„„´³named³port´³atom³ SignedInteger„„´³named³method´³refµ„³ MethodPattern„„´³named³path´³refµ„³ PathPattern„„´³named³handler´³embedded´³refµ„³ HttpRequest„„„„„„³ HttpContext´³rec´³lit³request„´³tupleµ´³named³req´³refµ„³ HttpRequest„„´³named³res´³embedded´³refµ„³ HttpResponse„„„„„„³ HttpRequest´³rec´³lit³ http-request„´³tupleµ´³named³sequenceNumber´³atom³ SignedInteger„„´³named³host´³atom³String„„´³named³port´³atom³ SignedInteger„„´³named³method´³atom³Symbol„„´³named³path´³seqof´³atom³String„„„´³named³headers´³refµ„³Headers„„´³named³query´³dictof´³atom³Symbol„´³seqof´³refµ„³
QueryValue„„„„´³named³body´³refµ„³ RequestBody„„„„„³ HttpService´³rec´³lit³ http-service„´³tupleµ´³named³host´³refµ„³ HostPattern„„´³named³port´³atom³ SignedInteger„„´³named³method´³refµ„³ MethodPattern„„´³named³path´³refµ„³ PathPattern„„„„„³ PathPattern´³seqof´³refµ„³PathPatternElement„„³ RequestBody´³orµµ±present´³atom³
ByteString„„µ±absent´³lit€„„„„³ HttpListener´³rec´³lit³ http-listener„´³tupleµ´³named³port´³atom³ SignedInteger„„„„„³ HttpResponse´³orµµ±status´³rec´³lit³status„´³tupleµ´³named³code´³atom³ SignedInteger„„´³named³message´³atom³String„„„„„„µ±header´³rec´³lit³header„´³tupleµ´³named³name´³atom³Symbol„„´³named³value´³atom³String„„„„„„µ±chunk´³rec´³lit³chunk„´³tupleµ´³named³chunk´³refµ„³Chunk„„„„„„µ±done´³rec´³lit³done„´³tupleµ´³named³chunk´³refµ„³Chunk„„„„„„„„³ MethodPattern´³orµµ±any´³lit€„„µ±specific´³atom³Symbol„„„„³PathPatternElement´³orµµ±label´³atom³String„„µ±wildcard´³lit³_„„µ±rest´³lit³...„„„„„³ embeddedType€„„µ³noise„´³schema·³version‘³ definitions·³Packet´³orµµ±complete´³atom³
ByteString„„µ±
fragmented´³seqof´³atom³
ByteString„„„„„³ NoiseSpec´³andµ´³dict·³key´³named³key´³atom³
ByteString„„³service´³named³service´³refµ„³ServiceSelector„„„„´³named³protocol´³refµ„³ NoiseProtocol„„´³named³ preSharedKeys´³refµ„³NoisePreSharedKeys„„„„³ NoiseProtocol´³orµµ±present´³dict·³protocol´³named³protocol´³atom³String„„„„„µ±invalid´³dict·³protocol´³named³protocol³any„„„„µ±absent´³dict·„„„„„³ NoiseStepType´³lit³noise„³SecretKeyField´³orµµ±present´³dict·³ secretKey´³named³ secretKey´³atom³
ByteString„„„„„µ±invalid´³dict·³ secretKey´³named³ secretKey³any„„„„µ±absent´³dict·„„„„„³DefaultProtocol´³lit±!Noise_NK_25519_ChaChaPoly_BLAKE2s„³NoiseStepDetail´³refµ„³ServiceSelector„³ServiceSelector³any³NoiseServiceSpec´³andµ´³named³base´³refµ„³ NoiseSpec„„´³named³ secretKey´³refµ„³SecretKeyField„„„„³NoisePreSharedKeys´³orµµ±present´³dict·³ preSharedKeys´³named³ preSharedKeys´³seqof´³atom³
ByteString„„„„„„µ±invalid´³dict·³ preSharedKeys´³named³ preSharedKeys³any„„„„µ±absent´³dict·„„„„„³NoisePathStepDetail´³refµ„³ NoiseSpec„³NoiseDescriptionDetail´³refµ„³NoiseServiceSpec„„³ embeddedType€„„µ³timer„´³schema·³version‘³ definitions·³SetTimer´³rec´³lit³ set-timer„´³tupleµ´³named³label³any„´³named³seconds´³atom³Double„„´³named³kind´³refµ„³ TimerKind„„„„„³ LaterThan´³rec´³lit³
later-than„´³tupleµ´³named³seconds´³atom³Double„„„„„³ TimerKind´³orµµ±relative´³lit³relative„„µ±absolute´³lit³absolute„„µ±clear´³lit³clear„„„„³ TimerExpired´³rec´³lit³ timer-expired„´³tupleµ´³named³label³any„´³named³seconds´³atom³Double„„„„„„³ embeddedType€„„µ³trace„´³schema·³version‘³ definitions·³Oid³any³Name´³orµµ± anonymous´³rec´³lit³ anonymous„´³tupleµ„„„„µ±named´³rec´³lit³named„´³tupleµ´³named³name³any„„„„„„„³Target´³rec´³lit³entity„´³tupleµ´³named³actor´³refµ„³ActorId„„´³named³facet´³refµ„³FacetId„„´³named³oid´³refµ„³Oid„„„„„³TaskId³any³TurnId³any³ActorId³any³FacetId³any³ TurnCause´³orµµ±turn´³rec´³lit³ caused-by„´³tupleµ´³named³id´³refµ„³TurnId„„„„„„µ±cleanup´³rec´³lit³cleanup„´³tupleµ„„„„µ±linkedTaskRelease´³rec´³lit³linked-task-release„´³tupleµ´³named³id´³refµ„³TaskId„„´³named³reason´³refµ„³LinkedTaskReleaseReason„„„„„„µ±periodicActivation´³rec´³lit³periodic-activation„´³tupleµ´³named³period´³atom³Double„„„„„„µ±delay´³rec´³lit³delay„´³tupleµ´³named³ causingTurn´³refµ„³TurnId„„´³named³amount´³atom³Double„„„„„„µ±external´³rec´³lit³external„´³tupleµ´³named³ description³any„„„„„„„³ TurnEvent´³orµµ±assert´³rec´³lit³assert„´³tupleµ´³named³ assertion´³refµ„³AssertionDescription„„´³named³handle´³refµ³protocol„³Handle„„„„„„µ±retract´³rec´³lit³retract„´³tupleµ´³named³handle´³refµ³protocol„³Handle„„„„„„µ±message´³rec´³lit³message„´³tupleµ´³named³body´³refµ„³AssertionDescription„„„„„„µ±sync´³rec´³lit³sync„´³tupleµ´³named³peer´³refµ„³Target„„„„„„µ± breakLink´³rec´³lit³
break-link„´³tupleµ´³named³source´³refµ„³ActorId„„´³named³handle´³refµ³protocol„³Handle„„„„„„„„³
ExitStatus´³orµµ±ok´³lit³ok„„µ±Error´³refµ³protocol„³Error„„„„³
TraceEntry´³rec´³lit³trace„´³tupleµ´³named³ timestamp´³atom³Double„„´³named³actor´³refµ„³ActorId„„´³named³item´³refµ„³ActorActivation„„„„„³ActorActivation´³orµµ±start´³rec´³lit³start„´³tupleµ´³named³ actorName´³refµ„³Name„„„„„„µ±turn´³refµ„³TurnDescription„„µ±stop´³rec´³lit³stop„´³tupleµ´³named³status´³refµ„³
ExitStatus„„„„„„„„³FacetStopReason´³orµµ±explicitAction´³lit³explicit-action„„µ±inert´³lit³inert„„µ±parentStopping´³lit³parent-stopping„„µ± actorStopping´³lit³actor-stopping„„„„³TurnDescription´³rec´³lit³turn„´³tupleµ´³named³id´³refµ„³TurnId„„´³named³cause´³refµ„³ TurnCause„„´³named³actions´³seqof´³refµ„³ActionDescription„„„„„„³ActionDescription´³orµµ±dequeue´³rec´³lit³dequeue„´³tupleµ´³named³event´³refµ„³TargetedTurnEvent„„„„„„µ±enqueue´³rec´³lit³enqueue„´³tupleµ´³named³event´³refµ„³TargetedTurnEvent„„„„„„µ±dequeueInternal´³rec´³lit³dequeue-internal„´³tupleµ´³named³event´³refµ„³TargetedTurnEvent„„„„„„µ±enqueueInternal´³rec´³lit³enqueue-internal„´³tupleµ´³named³event´³refµ„³TargetedTurnEvent„„„„„„µ±spawn´³rec´³lit³spawn„´³tupleµ´³named³link´³atom³Boolean„„´³named³id´³refµ„³ActorId„„„„„„µ±link´³rec´³lit³link„´³tupleµ´³named³ parentActor´³refµ„³ActorId„„´³named³ childToParent´³refµ³protocol„³Handle„„´³named³
childActor´³refµ„³ActorId„„´³named³ parentToChild´³refµ³protocol„³Handle„„„„„„µ±
facetStart´³rec´³lit³ facet-start„´³tupleµ´³named³path´³seqof´³refµ„³FacetId„„„„„„„µ± facetStop´³rec´³lit³
facet-stop„´³tupleµ´³named³path´³seqof´³refµ„³FacetId„„„´³named³reason´³refµ„³FacetStopReason„„„„„„µ±linkedTaskStart´³rec´³lit³linked-task-start„´³tupleµ´³named³taskName´³refµ„³Name„„´³named³id´³refµ„³TaskId„„„„„„„„³TargetedTurnEvent´³rec´³lit³event„´³tupleµ´³named³target´³refµ„³Target„„´³named³detail´³refµ„³ TurnEvent„„„„„³AssertionDescription´³orµµ±value´³rec´³lit³value„´³tupleµ´³named³value³any„„„„„µ±opaque´³rec´³lit³opaque„´³tupleµ´³named³ description³any„„„„„„„³LinkedTaskReleaseReason´³orµµ± cancelled´³lit³ cancelled„„µ±normal´³lit³normal„„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„µ³stream„´³schema·³version‘³ definitions·³Mode´³orµµ±bytes´³lit³bytes„„µ±lines´³refµ„³LineMode„„µ±packet´³rec´³lit³packet„´³tupleµ´³named³size´³atom³ SignedInteger„„„„„„µ±object´³rec´³lit³object„´³tupleµ´³named³ description³any„„„„„„„³Sink´³orµµ±source´³rec´³lit³source„´³tupleµ´³named³
controller´³embedded´³refµ„³Source„„„„„„„µ± StreamError´³refµ„³ StreamError„„µ±data´³rec´³lit³data„´³tupleµ´³named³payload³any„´³named³mode´³refµ„³Mode„„„„„„µ±eof´³rec´³lit³eof„´³tupleµ„„„„„„³Source´³orµµ±sink´³rec´³lit³sink„´³tupleµ´³named³
controller´³embedded´³refµ„³Sink„„„„„„„µ± StreamError´³refµ„³ StreamError„„µ±credit´³rec´³lit³credit„´³tupleµ´³named³amount´³refµ„³ CreditAmount„„´³named³mode´³refµ„³Mode„„„„„„„„³LineMode´³orµµ±lf´³lit³lf„„µ±crlf´³lit³crlf„„„„³ StreamError´³rec´³lit³error„´³tupleµ´³named³message´³atom³String„„„„„³ CreditAmount´³orµµ±count´³atom³ SignedInteger„„µ± unbounded´³lit³ unbounded„„„„³StreamConnection´³rec´³lit³stream-connection„´³tupleµ´³named³source´³embedded´³refµ„³Source„„„´³named³sink´³embedded´³refµ„³Sink„„„´³named³spec³any„„„„³StreamListenerError´³rec´³lit³stream-listener-error„´³tupleµ´³named³spec³any„´³named³message´³atom³String„„„„„³StreamListenerReady´³rec´³lit³stream-listener-ready„´³tupleµ´³named³spec³any„„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„µ³sturdy„´³schema·³version‘³ definitions·³Lit´³rec´³lit³lit„´³tupleµ´³named³value³any„„„„³Oid´³atom³ SignedInteger„³Alts´³rec´³lit³or„´³tupleµ´³named³ alternatives´³seqof´³refµ„³Rewrite„„„„„„³PAnd´³rec´³lit³and„´³tupleµ´³named³patterns´³seqof´³refµ„³Pattern„„„„„„³PNot´³rec´³lit³not„´³tupleµ´³named³pattern´³refµ„³Pattern„„„„„³TRef´³rec´³lit³ref„´³tupleµ´³named³binding´³atom³ SignedInteger„„„„„³PAtom´³orµµ±Boolean´³lit³Boolean„„µ±Float´³lit³Float„„µ±Double´³lit³Double„„µ± SignedInteger´³lit³ SignedInteger„„µ±String´³lit³String„„µ±
ByteString´³lit³
ByteString„„µ±Symbol´³lit³Symbol„„„„³PBind´³rec´³lit³bind„´³tupleµ´³named³pattern´³refµ„³Pattern„„„„„³Caveat´³orµµ±Rewrite´³refµ„³Rewrite„„µ±Alts´³refµ„³Alts„„µ±Reject´³refµ„³Reject„„µ±unknown³any„„„³Reject´³rec´³lit³reject„´³tupleµ´³named³pattern´³refµ„³Pattern„„„„„³Pattern´³orµµ±PDiscard´³refµ„³PDiscard„„µ±PAtom´³refµ„³PAtom„„µ± PEmbedded´³refµ„³ PEmbedded„„µ±PBind´³refµ„³PBind„„µ±PAnd´³refµ„³PAnd„„µ±PNot´³refµ„³PNot„„µ±Lit´³refµ„³Lit„„µ± PCompound´³refµ„³ PCompound„„„„³Rewrite´³rec´³lit³rewrite„´³tupleµ´³named³pattern´³refµ„³Pattern„„´³named³template´³refµ„³Template„„„„„³WireRef´³orµµ±mine´³tupleµ´³lit<69>„´³named³oid´³refµ„³Oid„„„„„µ±yours´³ tuplePrefixµ´³lit‘„´³named³oid´³refµ„³Oid„„„´³named³ attenuation´³seqof´³refµ„³Caveat„„„„„„„³PDiscard´³rec´³lit³_„´³tupleµ„„„³Template´³orµµ±
TAttenuate´³refµ„³
TAttenuate„„µ±TRef´³refµ„³TRef„„µ±Lit´³refµ„³Lit„„µ± TCompound´³refµ„³ TCompound„„„„³ PCompound´³orµµ±rec´³rec´³lit³rec„´³tupleµ´³named³label³any„´³named³fields´³seqof´³refµ„³Pattern„„„„„„„µ±arr´³rec´³lit³arr„´³tupleµ´³named³items´³seqof´³refµ„³Pattern„„„„„„„µ±dict´³rec´³lit³dict„´³tupleµ´³named³entries´³dictof³any´³refµ„³Pattern„„„„„„„„„³ PEmbedded´³lit³Embedded„³ SturdyRef´³rec´³lit³ref„´³tupleµ´³named³
parameters´³refµ„³
Parameters„„„„„³ TCompound´³orµµ±rec´³rec´³lit³rec„´³tupleµ´³named³label³any„´³named³fields´³seqof´³refµ„³Template„„„„„„„µ±arr´³rec´³lit³arr„´³tupleµ´³named³items´³seqof´³refµ„³Template„„„„„„„µ±dict´³rec´³lit³dict„´³tupleµ´³named³entries´³dictof³any´³refµ„³Template„„„„„„„„„³
Parameters´³andµ´³dict·³oid´³named³oid³any„³sig´³named³sig´³atom³
ByteString„„„„´³named³caveats´³refµ„³ CaveatsField„„„„³
TAttenuate´³rec´³lit³ attenuate„´³tupleµ´³named³template´³refµ„³Template„„´³named³ attenuation´³seqof´³refµ„³Caveat„„„„„„³ CaveatsField´³orµµ±present´³dict·³caveats´³named³caveats´³seqof´³refµ„³Caveat„„„„„„µ±invalid´³dict·³caveats´³named³caveats³any„„„„µ±absent´³dict·„„„„„³SturdyStepType´³lit³ref„³SturdyStepDetail´³refµ„³
Parameters„³SturdyPathStepDetail´³refµ„³
Parameters„³SturdyDescriptionDetail´³dict·³key´³named³key´³atom³
ByteString„„³oid´³named³oid³any„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„µ³worker„´³schema·³version‘³ definitions·³Instance´³rec´³lit³Instance„´³tupleµ´³named³name´³atom³String„„´³named³argument³any„„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„µ³service„´³schema·³version‘³ definitions·³State´³orµµ±started´³lit³started„„µ±ready´³lit³ready„„µ±failed´³lit³failed„„µ±complete´³lit³complete„„µ± userDefined³any„„„³
RunService´³rec´³lit³ run-service„´³tupleµ´³named³ serviceName³any„„„„³ ServiceState´³rec´³lit³ service-state„´³tupleµ´³named³ serviceName³any„´³named³state´³refµ„³State„„„„„³ ServiceObject´³rec´³lit³service-object„´³tupleµ´³named³ serviceName³any„´³named³object³any„„„„³RequireService´³rec´³lit³require-service„´³tupleµ´³named³ serviceName³any„„„„³RestartService´³rec´³lit³restart-service„´³tupleµ´³named³ serviceName³any„„„„³ServiceDependency´³rec´³lit³
depends-on„´³tupleµ´³named³depender³any„´³named³dependee´³refµ„³ ServiceState„„„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„µ³protocol„´³schema·³version‘³ definitions·³Oid´³atom³ SignedInteger„³Sync´³rec´³lit³sync„´³tupleµ´³named³peer´³embedded´³lit<69>„„„„„„³Turn´³seqof´³refµ„³ TurnEvent„„³Error´³rec´³lit³error„´³tupleµ´³named³message´³atom³String„„´³named³detail³any„„„„³Event´³orµµ±Assert´³refµ„³Assert„„µ±Retract´³refµ„³Retract„„µ±Message´³refµ„³Message„„µ±Sync´³refµ„³Sync„„„„³Assert´³rec´³lit³assert„´³tupleµ´³named³ assertion´³refµ„³ Assertion„„´³named³handle´³refµ„³Handle„„„„„³Handle´³atom³ SignedInteger„³Packet´³orµµ±Turn´³refµ„³Turn„„µ±Error´³refµ„³Error„„µ± Extension´³refµ„³ Extension„„„„³Message´³rec´³lit³message„´³tupleµ´³named³body´³refµ„³ Assertion„„„„„³Retract´³rec´³lit³retract„´³tupleµ´³named³handle´³refµ„³Handle„„„„„³ Assertion³any³ Extension´³rec´³named³label³any„´³named³fields´³seqof³any„„„³ TurnEvent´³tupleµ´³named³oid´³refµ„³Oid„„´³named³event´³refµ„³Event„„„„„³ embeddedType€„„µ³ dataspace„´³schema·³version‘³ definitions·³Observe´³rec´³lit³Observe„´³tupleµ´³named³pattern´³refµ³dataspacePatterns„³Pattern„„´³named³observer´³embedded³any„„„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„µ³
gatekeeper„´³schema·³version‘³ definitions·³Bind´³rec´³lit³bind„´³tupleµ´³named³ description´³refµ„³ Description„„´³named³target´³embedded³any„„´³named³observer´³refµ„³ BindObserver„„„„„³Step´³rec´³named³stepType´³atom³Symbol„„´³tupleµ´³named³detail³any„„„„³Bound´³orµµ±bound´³rec´³lit³bound„´³tupleµ´³named³pathStep´³refµ„³PathStep„„„„„„µ±Rejected´³refµ„³Rejected„„„„³Route´³rec´³lit³route„´³ tuplePrefixµ´³named³
transports´³seqof³any„„„´³named³ pathSteps´³seqof´³refµ„³PathStep„„„„„³Resolve´³rec´³lit³resolve„´³tupleµ´³named³step´³refµ„³Step„„´³named³observer´³embedded´³refµ„³Resolved„„„„„„³PathStep´³rec´³named³stepType´³atom³Symbol„„´³tupleµ´³named³detail³any„„„„³Rejected´³rec´³lit³rejected„´³tupleµ´³named³detail³any„„„„³Resolved´³orµµ±accepted´³rec´³lit³accepted„´³tupleµ´³named³responderSession´³embedded³any„„„„„„µ±Rejected´³refµ„³Rejected„„„„³ Description´³rec´³named³stepType´³atom³Symbol„„´³tupleµ´³named³detail³any„„„„³ ResolvePath´³rec´³lit³ resolve-path„´³tupleµ´³named³route´³refµ„³Route„„´³named³addr³any„´³named³control´³embedded´³refµ„³TransportControl„„„´³named³resolved´³refµ„³Resolved„„„„„³ BindObserver´³orµµ±present´³embedded´³refµ„³Bound„„„µ±absent´³lit€„„„„³ForceDisconnect´³rec´³lit³force-disconnect„´³tupleµ„„„³ResolvedPathStep´³rec´³lit³ path-step„´³tupleµ´³named³origin´³embedded´³refµ„³Resolve„„„´³named³pathStep´³refµ„³PathStep„„´³named³resolved´³refµ„³Resolved„„„„„³TransportControl´³refµ„³ForceDisconnect„³TransportConnection´³rec´³lit³connect-transport„´³tupleµ´³named³addr³any„´³named³control´³embedded´³refµ„³TransportControl„„„´³named³resolved´³refµ„³Resolved„„„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„µ³transportAddress„´³schema·³version‘³ definitions·³Tcp´³rec´³lit³tcp„´³tupleµ´³named³host´³atom³String„„´³named³port´³atom³ SignedInteger„„„„„³Unix´³rec´³lit³unix„´³tupleµ´³named³path´³atom³String„„„„„³Stdio´³rec´³lit³stdio„´³tupleµ„„„³ WebSocket´³rec´³lit³ws„´³tupleµ´³named³url´³atom³String„„„„„„³ embeddedType€„„µ³dataspacePatterns„´³schema·³version‘³ definitions·³DLit´³rec´³lit³lit„´³tupleµ´³named³value´³refµ„³AnyAtom„„„„„³DBind´³rec´³lit³bind„´³tupleµ´³named³pattern´³refµ„³Pattern„„„„„³AnyAtom´³orµµ±bool´³atom³Boolean„„µ±float´³atom³Float„„µ±double´³atom³Double„„µ±int´³atom³ SignedInteger„„µ±string´³atom³String„„µ±bytes´³atom³
ByteString„„µ±symbol´³atom³Symbol„„µ±embedded´³embedded³any„„„„³Pattern´³orµµ±DDiscard´³refµ„³DDiscard„„µ±DBind´³refµ„³DBind„„µ±DLit´³refµ„³DLit„„µ± DCompound´³refµ„³ DCompound„„„„³DDiscard´³rec´³lit³_„´³tupleµ„„„³ DCompound´³orµµ±rec´³rec´³lit³rec„´³tupleµ´³named³label³any„´³named³fields´³seqof´³refµ„³Pattern„„„„„„„µ±arr´³rec´³lit³arr„´³tupleµ´³named³items´³seqof´³refµ„³Pattern„„„„„„„µ±dict´³rec´³lit³dict„´³tupleµ´³named³entries´³dictof³any´³refµ„³Pattern„„„„„„„„„„³ embeddedType´³refµ³ EntityRef„³Cap„„„„„

3
implementations/python/docs/text.md Normal file
View File

@ -0,0 +1,3 @@
# Human-readable text syntax
::: preserves.text

5
implementations/python/docs/values.md Normal file
View File

@ -0,0 +1,5 @@
# Representations of Values
{% include "python-representation.md" %}
::: preserves.values

28
implementations/python/mkdocs.yml Normal file
View File

@ -0,0 +1,28 @@
site_name: Python Preserves
site_dir: ../../python/dev
site_url: https://preserves.dev/python/latest/
repo_url: https://gitlab.com/preserves/preserves
theme:
name: material
extra:
version:
provider: mike
plugins:
- search
- mkdocstrings:
handlers:
python:
options:
merge_init_into_class: true
- macros:
include_dir: ../../_includes
- git-revision-date-localized:
enable_creation_date: true
markdown_extensions:
- admonition
- pymdownx.highlight
- pymdownx.inlinehilite
- pymdownx.snippets
- pymdownx.superfences
watch:
- preserves

66
implementations/python/preserves/__init__.py
View File

@ -1,4 +1,63 @@
'''```
import preserves
```
The main package re-exports a subset of the exports of its constituent modules:
- From [preserves.values][]:
- [Annotated][preserves.values.Annotated]
- [Embedded][preserves.values.Embedded]
- [Float][preserves.values.Float]
- [ImmutableDict][preserves.values.ImmutableDict]
- [Record][preserves.values.Record]
- [Symbol][preserves.values.Symbol]
- [annotate][preserves.values.annotate]
- [is_annotated][preserves.values.is_annotated]
- [preserve][preserves.values.preserve]
- [strip_annotations][preserves.values.strip_annotations]
- From [preserves.error][]:
- [DecodeError][preserves.error.DecodeError]
- [EncodeError][preserves.error.EncodeError]
- [ShortPacket][preserves.error.ShortPacket]
- From [preserves.binary][]:
- [Decoder][preserves.binary.Decoder]
- [Encoder][preserves.binary.Encoder]
- [canonicalize][preserves.binary.canonicalize]
- [decode][preserves.binary.decode]
- [decode_with_annotations][preserves.binary.decode_with_annotations]
- [encode][preserves.binary.encode]
- From [preserves.text][]:
- [Formatter][preserves.text.Formatter]
- [Parser][preserves.text.Parser]
- [parse][preserves.text.parse]
- [parse_with_annotations][preserves.text.parse_with_annotations]
- [stringify][preserves.text.stringify]
- From [preserves.compare][]:
- [cmp][preserves.compare.cmp]
- From [preserves.merge][]:
- [merge][preserves.merge.merge]
It also exports the [compare][preserves.compare] and [fold][preserves.fold] modules themselves,
permitting patterns like
```python
>>> from preserves import *
>>> compare.cmp(123, 234)
-1
```
Finally, it provides a few utility aliases for common tasks:
'''
from .values import Float, Symbol, Record, ImmutableDict, Embedded, preserve
from .values import Annotated, is_annotated, strip_annotations, annotate
from .compare import cmp
@ -13,4 +72,11 @@ from .merge import merge
from . import fold, compare
loads = parse
'''
This alias for `parse` provides a familiar pythonesque name for converting a string to a Preserves `Value`.
'''
dumps = stringify
'''
This alias for `stringify` provides a familiar pythonesque name for converting a Preserves `Value` to a string.
'''

197
implementations/python/preserves/binary.py
View File

@ -1,3 +1,20 @@
"""The [preserves.binary][] module implements the [Preserves machine-oriented binary
syntax](https://preserves.dev/preserves-binary.html).
The main entry points are functions [encode][preserves.binary.encode],
[canonicalize][preserves.binary.canonicalize], [decode][preserves.binary.decode], and
[decode_with_annotations][preserves.binary.decode_with_annotations].
```python
>>> encode(Record(Symbol('hi'), []))
b'\\xb4\\xb3\\x02hi\\x84'
>>> decode(b'\\xb4\\xb3\\x02hi\\x84')
#hi()
```
"""
import numbers
import struct
@ -5,9 +22,108 @@ from .values import *
from .error import *
from .compat import basestring_, ord_
class BinaryCodec(object): pass
class BinaryCodec(object):
pass
class Decoder(BinaryCodec):
"""Implementation of a decoder for the machine-oriented binary Preserves syntax.
Args:
packet (bytes):
initial contents of the input buffer; may subsequently be extended by calling
[extend][preserves.binary.Decoder.extend].
include_annotations (bool):
if `True`, wrap each value and subvalue in an
[Annotated][preserves.values.Annotated] object.
decode_embedded:
function accepting a `Value` and returning a possibly-decoded form of that value
suitable for placing into an [Embedded][preserves.values.Embedded] object.
Normal usage is to supply a buffer, and keep calling [next][preserves.binary.Decoder.next]
until a [ShortPacket][preserves.error.ShortPacket] exception is raised:
```python
>>> d = Decoder(b'\\xa0{\\xb1\\x05hello\\x85\\xb3\\x01x\\xb5\\x84')
>>> d.next()
123
>>> d.next()
'hello'
>>> d.next()
()
>>> d.next()
Traceback (most recent call last):
...
preserves.error.ShortPacket: Short packet
```
Alternatively, keep calling [try_next][preserves.binary.Decoder.try_next] until it yields
`None`, which is not in the domain of Preserves `Value`s:
```python
>>> d = Decoder(b'\\xa0{\\xb1\\x05hello\\x85\\xb3\\x01x\\xb5\\x84')
>>> d.try_next()
123
>>> d.try_next()
'hello'
>>> d.try_next()
()
>>> d.try_next()
```
For convenience, [Decoder][preserves.binary.Decoder] implements the iterator interface,
backing it with [try_next][preserves.binary.Decoder.try_next], so you can simply iterate
over all complete values in an input:
```python
>>> d = Decoder(b'\\xa0{\\xb1\\x05hello\\x85\\xb3\\x01x\\xb5\\x84')
>>> list(d)
[123, 'hello', ()]
```
```python
>>> for v in Decoder(b'\\xa0{\\xb1\\x05hello\\x85\\xb3\\x01x\\xb5\\x84'):
... print(repr(v))
123
'hello'
()
```
Supply `include_annotations=True` to read annotations alongside the annotated values:
```python
>>> d = Decoder(b'\\xa0{\\xb1\\x05hello\\x85\\xb3\\x01x\\xb5\\x84', include_annotations=True)
>>> list(d)
[123, 'hello', @#x ()]
```
If you are incrementally reading from, say, a socket, you can use
[extend][preserves.binary.Decoder.extend] to add new input as if comes available:
```python
>>> d = Decoder(b'\\xa0{\\xb1\\x05he')
>>> d.try_next()
123
>>> d.try_next() # returns None because the input is incomplete
>>> d.extend(b'llo')
>>> d.try_next()
'hello'
>>> d.try_next()
```
Attributes:
packet (bytes): buffered input waiting to be processed
index (int): read position within `packet`
"""
def __init__(self, packet=b'', include_annotations=False, decode_embedded=lambda x: x):
super(Decoder, self).__init__()
self.packet = packet
@ -16,6 +132,8 @@ class Decoder(BinaryCodec):
self.decode_embedded = decode_embedded
def extend(self, data):
"""Appends `data` to the remaining bytes in `self.packet`, trimming already-processed
bytes from the front of `self.packet` and resetting `self.index` to zero."""
self.packet = self.packet[self.index:] + data
self.index = 0
@ -69,6 +187,11 @@ class Decoder(BinaryCodec):
return v
def next(self):
"""Reads the next complete `Value` from the internal buffer, raising
[ShortPacket][preserves.error.ShortPacket] if too few bytes are available, or
[DecodeError][preserves.error.DecodeError] if the input is invalid somehow.
"""
tag = self.nextbyte()
if tag == 0x80: return self.wrap(False)
if tag == 0x81: return self.wrap(True)
@ -99,6 +222,8 @@ class Decoder(BinaryCodec):
raise DecodeError('Invalid tag: ' + hex(tag))
def try_next(self):
"""Like [next][preserves.binary.Decoder.next], but returns `None` instead of raising
[ShortPacket][preserves.error.ShortPacket]."""
start = self.index
try:
return self.next()
@ -116,19 +241,71 @@ class Decoder(BinaryCodec):
return v
def decode(bs, **kwargs):
"""Yields the first complete encoded value from `bs`, passing `kwargs` through to the
[Decoder][preserves.binary.Decoder] constructor. Raises exceptions as per
[next][preserves.binary.Decoder.next].
Args:
bs (bytes): encoded data to decode
"""
return Decoder(packet=bs, **kwargs).next()
def decode_with_annotations(bs, **kwargs):
"""Like [decode][preserves.binary.decode], but supplying `include_annotations=True` to the
[Decoder][preserves.binary.Decoder] constructor."""
return Decoder(packet=bs, include_annotations=True, **kwargs).next()
class Encoder(BinaryCodec):
def __init__(self, encode_embedded=lambda x: x, canonicalize=False):
"""Implementation of an encoder for the machine-oriented binary Preserves syntax.
```python
>>> e = Encoder()
>>> e.append(123)
>>> e.append('hello')
>>> e.append(annotate([], Symbol('x')))
>>> e.contents()
b'\\xa0{\\xb1\\x05hello\\x85\\xb3\\x01x\\xb5\\x84'
```
Args:
encode_embedded:
function accepting an [Embedded][preserves.values.Embedded].embeddedValue and
returning a `Value` for serialization.
canonicalize (bool):
if `True`, ensures the serialized data are in [canonical
form](https://preserves.dev/canonical-binary.html). This is slightly more work than
producing potentially-non-canonical output.
include_annotations (bool | None):
if `None`, includes annotations in the output only when `canonicalize` is `False`,
because [canonical serialization of values demands omission of
annotations](https://preserves.dev/canonical-binary.html). If explicitly `True` or
`False`, however, annotations will be included resp. excluded no matter the
`canonicalize` setting. This can be used to get canonical ordering
(`canonicalize=True`) *and* annotations (`include_annotations=True`).
Attributes:
buffer (bytearray): accumulator for the output of the encoder
"""
def __init__(self,
encode_embedded=lambda x: x,
canonicalize=False,
include_annotations=None):
super(Encoder, self).__init__()
self.buffer = bytearray()
self._encode_embedded = encode_embedded
self._canonicalize = canonicalize
if include_annotations is None:
self.include_annotations = not self._canonicalize
else:
self.include_annotations = include_annotations
def reset(self):
"""Clears `self.buffer` to a fresh empty `bytearray`."""
self.buffer = bytearray()
def encode_embedded(self, v):
@ -137,6 +314,7 @@ class Encoder(BinaryCodec):
return self._encode_embedded(v)
def contents(self):
"""Returns a `bytes` constructed from the contents of `self.buffer`."""
return bytes(self.buffer)
def varint(self, v):
@ -174,7 +352,7 @@ class Encoder(BinaryCodec):
if not self._canonicalize:
self.encodevalues(6, v)
else:
c = Canonicalizer(self._encode_embedded)
c = Canonicalizer(self._encode_embedded, self.include_annotations)
for i in v: c.entry([i])
c.emit_entries(self, 6)
@ -182,11 +360,12 @@ class Encoder(BinaryCodec):
if not self._canonicalize:
self.encodevalues(7, list(dict_kvs(v)))
else:
c = Canonicalizer(self._encode_embedded)
c = Canonicalizer(self._encode_embedded, self.include_annotations)
for (kk, vv) in v.items(): c.entry([kk, vv])
c.emit_entries(self, 7)
def append(self, v):
"""Extend `self.buffer` with an encoding of `v`."""
v = preserve(v)
if hasattr(v, '__preserve_write_binary__'):
v.__preserve_write_binary__(self)
@ -230,8 +409,8 @@ class Encoder(BinaryCodec):
raise TypeError('Cannot preserves-encode: ' + repr(v))
class Canonicalizer:
def __init__(self, encode_embedded):
self.encoder = Encoder(encode_embedded, canonicalize=True)
def __init__(self, encode_embedded, include_annotations):
self.encoder = Encoder(encode_embedded, canonicalize=True, include_annotations=include_annotations)
self.entries = []
def entry(self, pieces):
@ -246,9 +425,15 @@ class Canonicalizer:
outer_encoder.buffer.append(0x84)
def encode(v, **kwargs):
"""Encode a single `Value` `v` to a byte string. Any supplied `kwargs` are passed on to the
underlying [Encoder][preserves.binary.Encoder] constructor."""
e = Encoder(**kwargs)
e.append(v)
return e.contents()
def canonicalize(v, **kwargs):
"""As [encode][preserves.binary.encode], but sets `canonicalize=True` in the
[Encoder][preserves.binary.Encoder] constructor.
"""
return encode(v, canonicalize=True, **kwargs)

48
implementations/python/preserves/compare.py
View File

@ -1,3 +1,34 @@
"""Preserves specifies a [total ordering](https://preserves.dev/preserves.html#total-order) and
an [equivalence](https://preserves.dev/preserves.html#equivalence) between terms. The
[preserves.compare][] module implements the ordering and equivalence relations.
```python
>>> cmp("bzz", "c")
-1
>>> cmp(True, [])
-1
>>> lt("bzz", "c")
True
>>> eq("bzz", "c")
False
```
Note that the ordering relates more values than Python's built-in ordering:
```python
>>> [1, 2, 2] < [1, 2, "3"]
Traceback (most recent call last):
..
TypeError: '<' not supported between instances of 'int' and 'str'
>>> lt([1, 2, 2], [1, 2, "3"])
True
```
"""
import numbers
from enum import Enum
from functools import cmp_to_key
@ -50,15 +81,23 @@ def type_number(v):
return TypeNumber.SEQUENCE
def cmp(a, b):
"""Returns `-1` if `a` < `b`, or `0` if `a` = `b`, or `1` if `a` > `b` according to the
[Preserves total order](https://preserves.dev/preserves.html#total-order)."""
return _cmp(preserve(a), preserve(b))
def lt(a, b):
"""Returns `True` iff `a` < `b` according to the [Preserves total
order](https://preserves.dev/preserves.html#total-order)."""
return cmp(a, b) < 0
def le(a, b):
"""Returns `True` iff `a` ≤ `b` according to the [Preserves total
order](https://preserves.dev/preserves.html#total-order)."""
return cmp(a, b) <= 0
def eq(a, b):
"""Returns `True` iff `a` = `b` according to the [Preserves equivalence
relation](https://preserves.dev/preserves.html#equivalence)."""
return _eq(preserve(a), preserve(b))
key = cmp_to_key(cmp)
@ -66,9 +105,18 @@ _key = key
_sorted = sorted
def sorted(iterable, *, key=lambda x: x, reverse=False):
"""Returns a sorted list built from `iterable`, extracting a sort key using `key`, and
ordering according to the [Preserves total
order](https://preserves.dev/preserves.html#total-order). Directly analogous to the
[built-in Python `sorted`
routine](https://docs.python.org/3/library/functions.html#sorted), except uses the
Preserves order instead of Python's less-than relation.
"""
return _sorted(iterable, key=lambda x: _key(key(x)), reverse=reverse)
def sorted_items(d):
"""Given a dictionary `d`, yields a list of `(key, value)` tuples sorted by `key`."""
return sorted(d.items(), key=_item_key)
def _eq_sequences(aa, bb):

19
implementations/python/preserves/error.py
View File

@ -1,3 +1,16 @@
class DecodeError(ValueError): pass
class EncodeError(ValueError): pass
class ShortPacket(DecodeError): pass
"""The [preserves.error][] module exports various `Error` subclasses."""
class DecodeError(ValueError):
"""Raised whenever [preserves.binary.Decoder][] or [preserves.text.Parser][] detect invalid
input."""
pass
class EncodeError(ValueError):
"""Raised whenever [preserves.binary.Encoder][] or [preserves.text.Formatter][] are unable to proceed."""
pass
class ShortPacket(DecodeError):
"""Raised whenever [preserves.binary.Decoder][] or [preserves.text.Parser][] discover that
they want to read beyond the end of the currently-available input buffer in order to
completely read an encoded value."""
pass

12
implementations/python/preserves/fold.py
View File

@ -1,6 +1,18 @@
"""The [preserves.fold][] module exports various utilities for traversing compound `Value`s."""
from .values import ImmutableDict, dict_kvs, Embedded, Record
def map_embeddeds(f, v):
"""Returns an [equivalent][preserves.compare.eq] copy of `v`, except where each contained
[Embedded][preserves.values.Embedded] value is replaced by `f` applied to the Embedded's
`embeddedValue` attribute.
```python
>>> map_embeddeds(lambda w: Embedded(f'w={w}'), ['a', Embedded(123), {'z': 6.0}])
('a', #!'w=123', {'z': 6.0})
```
"""
def walk(v):
if isinstance(v, Embedded):
return f(v.embeddedValue)

40
implementations/python/preserves/merge.py
View File

@ -1,9 +1,13 @@
"""The [preserves.merge][] module exports various utilities for merging `Value`s."""
from .values import ImmutableDict, dict_kvs, Embedded, Record
def merge_embedded_id(a, b):
return a if a is b else None
def merge(v0, *vs, merge_embedded=None):
"""Repeatedly merges `v0` with each element in `vs` using [merge2][preserves.merge.merge2],
returning the final result. The `merge_embedded` parameter is passed on to merge2."""
v = v0
for vN in vs:
v = merge2(v, vN, merge_embedded=merge_embedded)
@ -17,6 +21,42 @@ def merge_seq(aa, bb, merge_embedded=None):
return [merge2(a, b, merge_embedded=merge_embedded) for (a, b) in zip(aa, bb)]
def merge2(a, b, merge_embedded=None):
"""Merges `a` and `b`, returning the result. Raises `ValueError` if, during the merge, a
pair of incompatible values is discovered.
If `a` and `b` are [Embedded][preserves.values.Embedded] objects, their `embeddedValue`s
are merged using `merge_embedded`, and the result is again wrapped in an
[Embedded][preserves.values.Embedded] object.
```python
>>> merge2(123, 234)
Traceback (most recent call last):
...
ValueError: Cannot merge items
>>> merge2(123, 123)
123
>>> merge2('hi', 0)
Traceback (most recent call last):
...
ValueError: Cannot merge items
>>> merge2([1, 2], [1, 2])
[1, 2]
>>> merge2([1, 2], [1, 3])
Traceback (most recent call last):
...
ValueError: Cannot merge items
>>> merge2({'a': 1, 'b': 2}, {'a': 1, 'c': 3})
{'a': 1, 'b': 2, 'c': 3}
>>> merge2({'a': 1, 'b': 2}, {'a': 10, 'c': 3})
Traceback (most recent call last):
...
ValueError: Cannot merge items
>>> merge2(Record('a', [1, {'x': 2}]), Record('a', [1, {'y': 3}]))
a(1, {'x': 2, 'y': 3})
```
"""
if a == b:
return a
if isinstance(a, (list, tuple)) and isinstance(b, (list, tuple)):

155
implementations/python/preserves/path.py
View File

@ -1,16 +1,139 @@
"""The [preserves.path][] module implements [Preserves
Path](https://preserves.dev/preserves-path.html).
Preserves Path is roughly analogous to
[XPath](https://www.w3.org/TR/2017/REC-xpath-31-20170321/), but for Preserves values: just as
XPath selects portions of an XML document, a Preserves Path uses *path expressions* to select
portions of a `Value`.
Use [parse][preserves.path.parse] to compile a path expression, and then use the
[exec][preserves.path.exec] method on the result to apply it to a given input:
```python
parse(PATH_EXPRESSION_STRING).exec(PRESERVES_VALUE)
-> SEQUENCE_OF_PRESERVES_VALUES
```
## Command-line usage
When [preserves.path][] is run as a `__main__` module, `sys.argv[1]` is
[parsed][preserves.path.parse], interpreted as a path expression, and
[run][preserves.path.exec] against [human-readable values][preserves.text] read from standard
input. Each matching result is passed to [stringify][preserves.text.stringify] and printed to
standard output.
## Examples
### Setup: Loading test data
The following examples use `testdata`:
```python
>>> with open('tests/samples.bin', 'rb') as f:
... testdata = decode_with_annotations(f.read())
```
Recall that `samples.bin` contains a binary-syntax form of the human-readable
[`samples.pr](https://preserves.dev/tests/samples.pr) test data file, intended to exercise most
of the features of Preserves. In particular, the root `Value` in the file has a number of
annotations (for documentation and other purposes).
### Example 1: Selecting string-valued documentation annotations
The path expression `.annotations ^ Documentation . 0 / string` proceeds in five steps:
1. `.annotations` selects each annotation on the root document
2. `^ Documentation` retains only those values (each an annotation of the root) that are `Record`s with label equal to the symbol `Documentation`
3. `. 0` moves into the first child (the first field) of each such `Record`, which in our case is a list of other `Value`s
4. `/` selects all immediate children of these lists
5. `string` retains only those values that are strings
The result of evaluating it on `testdata` is as follows:
```python
>>> selector = parse('.annotations ^ Documentation . 0 / string')
>>> for result in selector.exec(testdata):
... print(stringify(result))
"Individual test cases may be any of the following record types:"
"In each test, let value = strip(annotatedValue),"
" forward = value,"
" back = value,"
"except where test-case-specific values of `forward` and/or `back`"
"are provided by the executing harness, and check the following"
"numbered expectations according to the table above:"
"Implementations may vary in their treatment of the difference between expectations"
"13/14 and 16/17, depending on how they wish to treat end-of-stream conditions."
```
### Example 2: Selecting tests with Records as their annotatedValues
The path expression `// [.^ [= Test + = NondeterministicTest]] [. 1 rec]` proceeds in three steps:
1. `//` recursively decomposes the input, yielding all direct and indirect descendants of each input value
2. `[.^ [= Test + = NondeterministicTest]]` retains only those inputs (each a descendant of the root) that yield more than zero results when executed against the expression within the brackets:
1. `.^` selects only labels of values that are `Records`, filtering by type and transforming in a single step
2. `[= Test + = NondeterministicTest]` again filters by a path expression:
1. the infix `+` operator takes the *union* of matches of its arguments
2. the left-hand argument, `= Test` selects values (remember, record labels) equal to the symbol `Test`
3. the right-hand argument `= NondeterministicTest` selects values equal to `NondeterministicTest`
The result is thus all `Record`s anywhere inside `testdata` that have either `Test` or `NondeterministicTest` as their labels.
3. `[. 1 rec]` filters these `Record`s by another path expression:
1. `. 1` selects their second field (fields are numbered from 0)
2. `rec` retains only values that are `Record`s
Evaluating the expression against `testdata` yields the following:
```python
>>> selector = parse('// [.^ [= Test + = NondeterministicTest]] [. 1 rec]')
>>> for result in selector.exec(testdata):
... print(stringify(result))
<Test #[tLMHY2FwdHVyZbSzB2Rpc2NhcmSEhA==] <capture <discard>>>
<Test #[tLMHb2JzZXJ2ZbSzBXNwZWFrtLMHZGlzY2FyZIS0swdjYXB0dXJltLMHZGlzY2FyZISEhIQ=] <observe <speak <discard> <capture <discard>>>>>
<Test #[tLWzBnRpdGxlZLMGcGVyc29ukrMFdGhpbmeRhKBlsQlCbGFja3dlbGy0swRkYXRloQcdkpOEsQJEcoQ=] <[titled person 2 thing 1] 101 "Blackwell" <date 1821 2 3> "Dr">>
<Test #[tLMHZGlzY2FyZIQ=] <discard>>
<Test #[tJe1hIQ=] <7 []>>
<Test #[tLMHZGlzY2FyZLMIc3VycHJpc2WE] <discard surprise>>
<Test #[tLEHYVN0cmluZ5OUhA==] <"aString" 3 4>>
<Test #[tLSzB2Rpc2NhcmSEk5SE] <<discard> 3 4>>
<Test #[hbMCYXK0swFShbMCYWazAWaE] @ar <R @af f>>
<Test #[tIWzAmFyswFShbMCYWazAWaE] <@ar R @af f>>
```
"""
from . import *
from .schema import load_schema_file, extend
from .values import _unwrap
from .compat import basestring_
from . import compare as preserves_compare
import pathlib
import re
syntax = load_schema_file(pathlib.Path(__file__).parent / 'path.prb').path
"""This value is a Python representation of a [Preserves Schema][preserves.schema] definition
for the Preserves Path expression language. The language is defined in the file
[path.prs](https://preserves.dev/path/path.prs)."""
Selector = syntax.Selector
"""Schema definition for representing a sequence of Preserves Path `Step`s."""
Predicate = syntax.Predicate
"""Schema definition for representing a Preserves Path `Predicate`."""
def parse(s):
"""Parse `s` as a Preserves Path path expression, yielding a
[Selector][preserves.path.Selector] object. Selectors (and Predicates etc.) have an
[exec][preserves.path.exec] method defined on them.
Raises `ValueError` if `s` is not a valid path expression.
"""
return parse_selector(Parser(s))
def parse_selector(tokens):
@ -270,6 +393,7 @@ def exec(self, v):
@extend(syntax.Axis.embedded)
def exec(self, v):
v = preserve(_unwrap(v))
return (v.embeddedValue,) if isinstance(v, Embedded) else ()
@extend(syntax.Filter.nop)
@ -278,35 +402,37 @@ def exec(self, v):
@extend(syntax.Filter.compare)
def exec(self, v):
v = preserve(_unwrap(v))
return (v,) if self.op.compare(v, self.literal) else ()
@extend(syntax.Comparison.eq)
def compare(self, lhs, rhs):
return lhs == rhs
return preserves_compare.eq(lhs, rhs)
@extend(syntax.Comparison.ne)
def compare(self, lhs, rhs):
return lhs != rhs
return not preserves_compare.eq(lhs, rhs)
@extend(syntax.Comparison.lt)
def compare(self, lhs, rhs):
return lhs < rhs
return preserves_compare.lt(lhs, rhs)
@extend(syntax.Comparison.ge)
def compare(self, lhs, rhs):
return lhs >= rhs
return not preserves_compare.lt(lhs, rhs)
@extend(syntax.Comparison.gt)
def compare(self, lhs, rhs):
return lhs > rhs
return not preserves_compare.le(lhs, rhs)
@extend(syntax.Comparison.le)
def compare(self, lhs, rhs):
return lhs <= rhs
return preserves_compare.le(lhs, rhs)
@extend(syntax.Filter.regex)
def exec(self, v):
r = re.compile(self.regex)
v = preserve(_unwrap(v))
if isinstance(v, Symbol):
return (v,) if r.match(v.name) else ()
if isinstance(v, basestring_):
@ -319,6 +445,7 @@ def exec(self, v):
@extend(syntax.Filter.real)
def exec(self, v):
v = preserve(_unwrap(v))
if isinstance(v, Float):
return (v.value,)
if type(v) == float:
@ -329,6 +456,7 @@ def exec(self, v):
@extend(syntax.Filter.int)
def exec(self, v):
v = preserve(_unwrap(v))
if isinstance(v, Float):
return (int(v.value()),)
if type(v) == float:
@ -339,6 +467,7 @@ def exec(self, v):
@extend(syntax.Filter.kind)
def exec(self, v):
v = preserve(_unwrap(v))
return self.kind.exec(v)
@extend(syntax.ValueKind.Boolean)
@ -391,8 +520,22 @@ def exec(self, v):
@extend(syntax.Function)
def exec(self, v):
"""WARNING: This is not a *function*: it is a *method* on
[Selector][preserves.path.Selector], [Predicate][preserves.path.Predicate], and so on.
```python
>>> sel = parse('/ [.length gt 1]')
>>> sel.exec(['', 'a', 'ab', 'abc', 'abcd', 'bcd', 'cd', 'd', ''])
('ab', 'abc', 'abcd', 'bcd', 'cd')
```
"""
return (len(self.selector.exec(v)),)
### NOTE WELL: the *LAST* definition of exec in this file is the one that needs the docstring
### attached!
if __name__ == '__main__':
import sys
sel = parse(sys.argv[1])

2
implementations/python/preserves/schema.prb
View File

@ -5,4 +5,4 @@ ByteString
ByteString„„µ±Symbol´³lit³Symbol„„„„³
Definition´³orµµ±or´³rec´³lit³or„´³tupleµ´³ tuplePrefixµ´³named³pattern0´³refµ„³NamedAlternative„„´³named³pattern1´³refµ„³NamedAlternative„„„´³named³patternN´³seqof´³refµ„³NamedAlternative„„„„„„„„µ±and´³rec´³lit³and„´³tupleµ´³ tuplePrefixµ´³named³pattern0´³refµ„³ NamedPattern„„´³named³pattern1´³refµ„³ NamedPattern„„„´³named³patternN´³seqof´³refµ„³ NamedPattern„„„„„„„„µ±Pattern´³refµ„³Pattern„„„„³
ModulePath´³seqof´³atom³Symbol„„³ Definitions´³dictof´³atom³Symbol„´³refµ„³
Definition„„³ NamedPattern´³orµµ±named´³refµ„³Binding„„µ± anonymous´³refµ„³Pattern„„„„³ SimplePattern´³orµµ±any´³lit³any„„µ±atom´³rec´³lit³atom„´³tupleµ´³named³atomKind´³refµ„³AtomKind„„„„„„µ±embedded´³rec´³lit³embedded„´³tupleµ´³named³ interface´³refµ„³ SimplePattern„„„„„„µ±lit´³rec´³lit³lit„´³tupleµ´³named³value³any„„„„„µ±seqof´³rec´³lit³seqof„´³tupleµ´³named³pattern´³refµ„³ SimplePattern„„„„„„µ±setof´³rec´³lit³setof„´³tupleµ´³named³pattern´³refµ„³ SimplePattern„„„„„„µ±dictof´³rec´³lit³dictof„´³tupleµ´³named³key´³refµ„³ SimplePattern„„´³named³value´³refµ„³ SimplePattern„„„„„„µ±Ref´³refµ„³Ref„„„„³CompoundPattern´³orµµ±rec´³rec´³lit³rec„´³tupleµ´³named³label´³refµ„³ NamedPattern„„´³named³fields´³refµ„³ NamedPattern„„„„„„µ±tuple´³rec´³lit³tuple„´³tupleµ´³named³patterns´³seqof´³refµ„³ NamedPattern„„„„„„„µ± tuplePrefix´³rec´³lit³ tuplePrefix„´³tupleµ´³named³fixed´³seqof´³refµ„³ NamedPattern„„„´³named³variable´³refµ„³NamedSimplePattern„„„„„„µ±dict´³rec´³lit³dict„´³tupleµ´³named³entries´³refµ„³DictionaryEntries„„„„„„„„³EmbeddedTypeName´³orµµ±Ref´³refµ„³Ref„„µ±false´³lit€„„„„³NamedAlternative´³tupleµ´³named³ variantLabel´³atom³String„„´³named³pattern´³refµ„³Pattern„„„„³DictionaryEntries´³dictof³any´³refµ„³NamedSimplePattern„„³NamedSimplePattern´³orµµ±named´³refµ„³Binding„„µ± anonymous´³refµ„³ SimplePattern„„„„„³ embeddedType€„„
Definition„„³ NamedPattern´³orµµ±named´³refµ„³Binding„„µ± anonymous´³refµ„³Pattern„„„„³ SimplePattern´³orµµ±any´³lit³any„„µ±atom´³rec´³lit³atom„´³tupleµ´³named³atomKind´³refµ„³AtomKind„„„„„„µ±embedded´³rec´³lit³embedded„´³tupleµ´³named³ interface´³refµ„³ SimplePattern„„„„„„µ±lit´³rec´³lit³lit„´³tupleµ´³named³value³any„„„„„µ±seqof´³rec´³lit³seqof„´³tupleµ´³named³pattern´³refµ„³ SimplePattern„„„„„„µ±setof´³rec´³lit³setof„´³tupleµ´³named³pattern´³refµ„³ SimplePattern„„„„„„µ±dictof´³rec´³lit³dictof„´³tupleµ´³named³key´³refµ„³ SimplePattern„„´³named³value´³refµ„³ SimplePattern„„„„„„µ±Ref´³refµ„³Ref„„„„³CompoundPattern´³orµµ±rec´³rec´³lit³rec„´³tupleµ´³named³label´³refµ„³ NamedPattern„„´³named³fields´³refµ„³ NamedPattern„„„„„„µ±tuple´³rec´³lit³tuple„´³tupleµ´³named³patterns´³seqof´³refµ„³ NamedPattern„„„„„„„µ± tuplePrefix´³rec´³lit³ tuplePrefix„´³tupleµ´³named³fixed´³seqof´³refµ„³ NamedPattern„„„´³named³variable´³refµ„³NamedSimplePattern„„„„„„µ±dict´³rec´³lit³dict„´³tupleµ´³named³entries´³refµ„³DictionaryEntries„„„„„„„„³EmbeddedTypeName´³orµµ±false´³lit€„„µ±Ref´³refµ„³Ref„„„„³NamedAlternative´³tupleµ´³named³ variantLabel´³atom³String„„´³named³pattern´³refµ„³Pattern„„„„³DictionaryEntries´³dictof³any´³refµ„³NamedSimplePattern„„³NamedSimplePattern´³orµµ±named´³refµ„³Binding„„µ± anonymous´³refµ„³ SimplePattern„„„„„³ embeddedType€„„

535
implementations/python/preserves/schema.py
View File

@ -1,11 +1,272 @@
#
# This is an implementation of [Preserves Schema](https://preserves.dev/preserves-schema.html)
# for Python 3.
#
"""The [preserves.schema][] module implements [Preserves
Schema](https://preserves.dev/preserves-schema.html) for Python.
A Schema source file (like [this one](https://preserves.dev/schema/schema.prs)) is first
compiled using [`preserves-schemac`](https://preserves.dev/doc/preserves-schemac.html) to
produce a binary-syntax *schema bundle* containing schema module definitons (like [this
one](https://preserves.dev/preserves-schema.html#appendix-metaschema-instance)). Python code
then loads the bundle, exposing its contents as [Namespace][preserves.schema.Namespace]s
ultimately containing [SchemaObject][preserves.schema.SchemaObject]s.
## Examples
### Setup: Loading a schema bundle
For our running example, we will use schemas associated with the [Syndicated Actor
Model](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols). (The schema bundle
is a copy of [this
file](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/branch/main/schema-bundle.bin)
from the `syndicate-protocols` repository.)
To load a schema bundle, use [load_schema_file][preserves.schema.load_schema_file] (or,
alternatively, use [Compiler][preserves.schema.Compiler] directly):
```python
>>> bundle = load_schema_file('docs/syndicate-protocols-schema-bundle.bin')
>>> type(bundle)
<class 'preserves.schema.Namespace'>
```
The top-level entries in the loaded bundle are schema modules. Let's examine the `stream`
schema module, whose [source
code](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/commit/d8a139b23a40bad6698f9f4240f9e8426b4a123f/schemas/stream.prs)
indicates that it should contain definitions for `Mode`, `Source`, `Sink`, etc.:
```python
>>> bundle.stream # doctest: +ELLIPSIS
{'Mode': <class 'stream.Mode'>, 'Sink': <class 'stream.Sink'>, ...}
```
### Example 1: stream.StreamListenerError, a product type
Drilling down further, let's consider the [definition of
StreamListenerError](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/commit/d8a139b23a40bad6698f9f4240f9e8426b4a123f/schemas/stream.prs#L9), which appears in the source as
```
StreamListenerError = <stream-listener-error @spec any @message string> .
```
This reads, in the [Preserves Schema
language](https://preserves.dev/preserves-schema.html#the-preserves-schema-language), as the
definition of a simple product type (record, class, object) with two named fields `spec` and
`message`. Parsing a value into a `StreamListenerError` will only succeed if it's a record, if
the label matches, the second field (`message`) is a string, and it has exactly two fields.
```python
>>> bundle.stream.StreamListenerError
<class 'stream.StreamListenerError'>
```
The `StreamListenerError` class includes a [decode][preserves.schema.SchemaObject.decode]
method that analyzes an input value:
```python
>>> bundle.stream.StreamListenerError.decode(
... parse('<stream-listener-error <xyz> "an error">'))
StreamListenerError {'spec': #xyz(), 'message': 'an error'}
```
If invalid input is supplied, [decode][preserves.schema.SchemaObject.decode] will raise
[SchemaDecodeFailed][preserves.schema.SchemaDecodeFailed], which includes helpful information
for diagnosing the problem (as we will see below, this is especially useful for parsers for sum
types):
```python
>>> bundle.stream.StreamListenerError.decode(
... parse('<i-am-invalid>'))
Traceback (most recent call last):
...
preserves.schema.SchemaDecodeFailed: Could not decode i-am-invalid using <class 'stream.StreamListenerError'>
Most likely reason: in stream.StreamListenerError: <lit stream-listener-error> didn't match i-am-invalid
Full explanation:
in stream.StreamListenerError: <lit stream-listener-error> didn't match i-am-invalid
```
Alternatively, the [try_decode][preserves.schema.SchemaObject.try_decode] method catches
[SchemaDecodeFailed][preserves.schema.SchemaDecodeFailed], transforming it into `None`:
```python
>>> bundle.stream.StreamListenerError.try_decode(
... parse('<stream-listener-error <xyz> "an error">'))
StreamListenerError {'spec': #xyz(), 'message': 'an error'}
>>> bundle.stream.StreamListenerError.try_decode(
... parse('<i-am-invalid>'))
```
The class can also be instantiated directly:
```python
>>> err = bundle.stream.StreamListenerError(Record(Symbol('xyz'), []), 'an error')
>>> err
StreamListenerError {'spec': #xyz(), 'message': 'an error'}
```
The fields and contents of instances can be queried:
```python
>>> err.spec
#xyz()
>>> err.message
'an error'
```
And finally, instances can of course be serialized and encoded:
```python
>>> print(stringify(err))
<stream-listener-error <xyz> "an error">
>>> canonicalize(err)
b'\\xb4\\xb3\\x15stream-listener-error\\xb4\\xb3\\x03xyz\\x84\\xb1\\x08an error\\x84'
```
### Example 2: stream.Mode, a sum type
Now let's consider the [definition of
Mode](https://git.syndicate-lang.org/syndicate-lang/syndicate-protocols/src/commit/d8a139b23a40bad6698f9f4240f9e8426b4a123f/schemas/stream.prs#L37),
which appears in the source as
```
Mode = =bytes / @lines LineMode / <packet @size int> / <object @description any> .
```
This reads, in the [Preserves Schema
language](https://preserves.dev/preserves-schema.html#the-preserves-schema-language), as an
alternation (disjoint union, variant, sum type) of four possible kinds of value: the symbol
`bytes`; a `LineMode` value; a record with `packet` as its label and an integer as its only
field; or a record with `object` as its label and any kind of value as its only field. In
Python, this becomes:
```python
>>> bundle.stream.Mode.bytes
<class 'stream.Mode.bytes'>
>>> bundle.stream.Mode.lines
<class 'stream.Mode.lines'>
>>> bundle.stream.Mode.packet
<class 'stream.Mode.packet'>
>>> bundle.stream.Mode.object
<class 'stream.Mode.object'>
```
As before, `Mode` includes a [decode][preserves.schema.SchemaObject.decode] method that analyzes
an input value:
```python
>>> bundle.stream.Mode.decode(parse('bytes'))
Mode.bytes()
>>> bundle.stream.Mode.decode(parse('lf'))
Mode.lines(LineMode.lf())
>>> bundle.stream.Mode.decode(parse('<packet 123>'))
Mode.packet {'size': 123}
>>> bundle.stream.Mode.decode(parse('<object "?">'))
Mode.object {'description': '?'}
```
Invalid input causes [SchemaDecodeFailed][preserves.schema.SchemaDecodeFailed] to be raised:
```python
>>> bundle.stream.Mode.decode(parse('<i-am-not-a-valid-mode>'))
Traceback (most recent call last):
...
preserves.schema.SchemaDecodeFailed: Could not decode <i-am-not-a-valid-mode> using <class 'stream.Mode'>
Most likely reason: in stream.LineMode.crlf: <lit crlf> didn't match <i-am-not-a-valid-mode>
Full explanation:
in stream.Mode: matching <i-am-not-a-valid-mode>
in stream.Mode.bytes: <lit bytes> didn't match <i-am-not-a-valid-mode>
in stream.Mode.lines: <ref [] LineMode> didn't match <i-am-not-a-valid-mode>
in stream.LineMode: matching <i-am-not-a-valid-mode>
in stream.LineMode.lf: <lit lf> didn't match <i-am-not-a-valid-mode>
in stream.LineMode.crlf: <lit crlf> didn't match <i-am-not-a-valid-mode>
in stream.Mode.packet: <lit packet> didn't match i-am-not-a-valid-mode
in stream.Mode.object: <lit object> didn't match i-am-not-a-valid-mode
```
The "full explanation" includes details on which parses were attempted, and why they failed.
Again, the [try_decode][preserves.schema.SchemaObject.try_decode] method catches
[SchemaDecodeFailed][preserves.schema.SchemaDecodeFailed], transforming it into `None`:
```python
>>> bundle.stream.Mode.try_decode(parse('bytes'))
Mode.bytes()
>>> bundle.stream.Mode.try_decode(parse('<i-am-not-a-valid-mode>'))
```
Direct instantiation is done with the variant classes, not with `Mode` itself:
```python
>>> bundle.stream.Mode.bytes()
Mode.bytes()
>>> bundle.stream.Mode.lines(bundle.stream.LineMode.lf())
Mode.lines(LineMode.lf())
>>> bundle.stream.Mode.packet(123)
Mode.packet {'size': 123}
>>> bundle.stream.Mode.object('?')
Mode.object {'description': '?'}
```
Fields and contents can be queried as usual:
```python
>>> bundle.stream.Mode.lines(bundle.stream.LineMode.lf()).value
LineMode.lf()
>>> bundle.stream.Mode.packet(123).size
123
>>> bundle.stream.Mode.object('?').description
'?'
```
And serialization and encoding are also as expected:
```python
>>> print(stringify(bundle.stream.Mode.bytes()))
bytes
>>> print(stringify(bundle.stream.Mode.lines(bundle.stream.LineMode.lf())))
lf
>>> print(stringify(bundle.stream.Mode.packet(123)))
<packet 123>
>>> print(stringify(bundle.stream.Mode.object('?')))
<object "?">
>>> canonicalize(bundle.stream.Mode.object('?'))
b'\\xb4\\xb3\\x06object\\xb1\\x01?\\x84'
```
Finally, the [VARIANT][preserves.schema.SchemaObject.VARIANT] attribute of instances
allows code to dispatch on what kind of data it is handling at a given moment:
```python
>>> bundle.stream.Mode.bytes().VARIANT
#bytes
>>> bundle.stream.Mode.lines(bundle.stream.LineMode.lf()).VARIANT
#lines
>>> bundle.stream.Mode.packet(123).VARIANT
#packet
>>> bundle.stream.Mode.object('?').VARIANT
#object
```
"""
from . import *
import pathlib
import keyword
from functools import wraps
AND = Symbol('and')
ANY = Symbol('any')
@ -38,6 +299,15 @@ def sequenceish(x):
return isinstance(x, tuple) or isinstance(x, list)
class SchemaDecodeFailed(ValueError):
"""Raised when [decode][preserves.schema.SchemaObject.decode] cannot find a way to parse a
given input.
Attributes:
cls (class): the SchemaObject subclass attempting the parse
pattern (Value): the failing pattern, a `Value` conforming to schema `meta.Pattern`
value (Value): the unparseable value
failures (list[SchemaDecodeFailed]): descriptions of failed paths attempted during the match this failure describes
"""
def __init__(self, cls, p, v, failures=None):
super().__init__()
self.cls = cls
@ -81,18 +351,108 @@ class ExplanationBuilder:
return '\n' + ' ' * self.indentLevel + self._node(failure) + ''.join(nested)
class SchemaObject:
"""Base class for classes representing grammatical productions in a schema: instances of
[SchemaObject][preserves.schema.SchemaObject] represent schema *definitions*. This is an
abstract class, as are its subclasses [Enumeration][preserves.schema.Enumeration] and
[Definition][preserves.schema.Definition]. It is subclasses of *those* subclasses,
automatically produced during schema loading, that are actually instantiated.
```python
>>> bundle = load_schema_file('docs/syndicate-protocols-schema-bundle.bin')
>>> bundle.stream.Mode.mro()[1:-1]
[<class 'preserves.schema.Enumeration'>, <class 'preserves.schema.SchemaObject'>]
>>> bundle.stream.Mode.packet.mro()[1:-1]
[<class 'stream.Mode._ALL'>, <class 'preserves.schema.Definition'>, <class 'preserves.schema.SchemaObject'>]
>>> bundle.stream.StreamListenerError.mro()[1:-1]
[<class 'preserves.schema.Definition'>, <class 'preserves.schema.SchemaObject'>]
```
Illustrating the class attributes on [SchemaObject][preserves.schema.SchemaObject]
subclasses:
```python
>>> bundle.stream.Mode.ROOTNS is bundle
True
>>> print(stringify(bundle.stream.Mode.SCHEMA, indent=2))
<or [
[
"bytes"
<lit bytes>
]
[
"lines"
<ref [] LineMode>
]
[
"packet"
<rec <lit packet> <tuple [<named size <atom SignedInteger>>]>>
]
[
"object"
<rec <lit object> <tuple [<named description any>]>>
]
]>
>>> bundle.stream.Mode.MODULE_PATH
(#stream,)
>>> bundle.stream.Mode.NAME
#Mode
>>> bundle.stream.Mode.VARIANT is None
True
>>> bundle.stream.Mode.packet.VARIANT
#packet
```
"""
ROOTNS = None
"""A [Namespace][preserves.schema.Namespace] that is the top-level environment for all
bundles included in the [Compiler][preserves.schema.Compiler] run that produced this
[SchemaObject][preserves.schema.SchemaObject].
"""
SCHEMA = None
"""A `Value` conforming to schema `meta.Definition` (and thus often to `meta.Pattern`
etc.), interpreted by the [SchemaObject][preserves.schema.SchemaObject] machinery to drive
parsing, unparsing and so forth."""
MODULE_PATH = None
"""A sequence (tuple) of [Symbol][preserves.values.Symbol]s naming the path from the root
to the schema module containing this definition."""
NAME = None
"""A [Symbol][preserves.values.Symbol] naming this definition within its module."""
VARIANT = None
"""`None` for [Definition][preserves.schema.Definition]s (such as
`bundle.stream.StreamListenerError` above) and for overall
[Enumeration][preserves.schema.Enumeration]s (such as `bundle.stream.Mode`), or a
[Symbol][preserves.values.Symbol] for variant definitions *contained within* an enumeration
(such as `bundle.stream.Mode.packet`).
"""
@classmethod
def decode(cls, v):
"""Parses `v` using the [SCHEMA][preserves.schema.SchemaObject.SCHEMA], returning a
(sub)instance of [SchemaObject][preserves.schema.SchemaObject] or raising
[SchemaDecodeFailed][preserves.schema.SchemaDecodeFailed]."""
raise NotImplementedError('Subclass responsibility')
@classmethod
def try_decode(cls, v):
"""Parses `v` using the [SCHEMA][preserves.schema.SchemaObject.SCHEMA], returning a
(sub)instance of [SchemaObject][preserves.schema.SchemaObject] or `None` if parsing
failed."""
try:
return cls.decode(v)
except SchemaDecodeFailed:
@ -176,6 +536,8 @@ class SchemaObject:
raise ValueError(f'Bad schema {p}')
def __preserve__(self):
"""Called by [preserves.values.preserve][]: *unparses* the information represented by
this instance, using its schema definition, to produce a Preserves `Value`."""
raise NotImplementedError('Subclass responsibility')
def __repr__(self):
@ -192,7 +554,29 @@ class SchemaObject:
raise NotImplementedError('Subclass responsibility')
class Enumeration(SchemaObject):
"""Subclasses of [Enumeration][preserves.schema.Enumeration] represent a group of variant
options within a sum type.
```python
>>> bundle = load_schema_file('docs/syndicate-protocols-schema-bundle.bin')
>>> import pprint
>>> pprint.pprint(bundle.stream.Mode.VARIANTS)
[(#bytes, <class 'stream.Mode.bytes'>),
(#lines, <class 'stream.Mode.lines'>),
(#packet, <class 'stream.Mode.packet'>),
(#object, <class 'stream.Mode.object'>)]
>>> bundle.stream.Mode.VARIANTS[0][1] is bundle.stream.Mode.bytes
True
```
"""
VARIANTS = None
"""List of `(Symbol, SchemaObject class)` tuples representing the possible options within
this sum type."""
def __init__(self):
raise TypeError('Cannot create instance of Enumeration')
@ -227,6 +611,7 @@ class Enumeration(SchemaObject):
raise TypeError('Cannot encode instance of Enumeration')
def safeattrname(k):
"""Escapes Python keywords by prepending `_`; passes all other strings through."""
return k + '_' if keyword.iskeyword(k) else k
def safesetattr(o, k, v):
@ -239,11 +624,61 @@ def safehasattr(o, k):
return hasattr(o, safeattrname(k))
class Definition(SchemaObject):
"""Subclasses of [Definition][preserves.schema.Definition] are used to represent both
standalone non-alternation definitions as well as alternatives within an
[Enumeration][preserves.schema.Enumeration].
```python
>>> bundle = load_schema_file('docs/syndicate-protocols-schema-bundle.bin')
>>> bundle.stream.StreamListenerError.FIELD_NAMES
['spec', 'message']
>>> bundle.stream.StreamListenerError.SAFE_FIELD_NAMES
['spec', 'message']
>>> bundle.stream.StreamListenerError.ENUMERATION is None
True
>>> bundle.stream.Mode.object.FIELD_NAMES
['description']
>>> bundle.stream.Mode.object.SAFE_FIELD_NAMES
['description']
>>> bundle.stream.Mode.object.ENUMERATION is bundle.stream.Mode
True
>>> bundle.stream.CreditAmount.count.FIELD_NAMES
[]
>>> bundle.stream.CreditAmount.count.SAFE_FIELD_NAMES
[]
>>> bundle.stream.CreditAmount.count.ENUMERATION is bundle.stream.CreditAmount
True
>>> bundle.stream.CreditAmount.decode(parse('123'))
CreditAmount.count(123)
>>> bundle.stream.CreditAmount.count(123)
CreditAmount.count(123)
>>> bundle.stream.CreditAmount.count(123).value
123
```
"""
EMPTY = False
SIMPLE = False
FIELD_NAMES = []
"""List of strings: names of the fields contained within this definition, if it has named
fields at all; otherwise, an empty list, and the definition is a simple wrapper for another
value, in which case that value is accessed via the `value` attribute."""
SAFE_FIELD_NAMES = []
"""The list produced by mapping [safeattrname][preserves.schema.safeattrname] over
[FIELD_NAMES][preserves.schema.Definition.FIELD_NAMES]."""
ENUMERATION = None
"""`None` for standalone top-level definitions with a module; otherwise, an
[Enumeration][preserves.schema.Enumeration] subclass representing a top-level alternation
definition."""
def _constructor_name(self):
if self.VARIANT is None:
@ -431,6 +866,13 @@ def definition_not_found(module_path, name):
raise KeyError('Definition not found: ' + module_path_str(module_path + (name,)))
class Namespace:
"""A [Namespace][preserves.schema.Namespace] is a dictionary-like object representing a
schema module that knows its location in a schema module hierarchy and whose attributes
correspond to definitions and submodules within the schema module.
Attributes:
_prefix (tuple[Symbol]): path to this module/Namespace from the root Namespace
"""
def __init__(self, prefix):
self._prefix = prefix
@ -453,18 +895,50 @@ class Namespace:
return repr(self._items())
class Compiler:
"""Instances of [Compiler][preserves.schema.Compiler] populate an initially-empty
[Namespace][preserves.schema.Namespace] by loading and compiling schema bundle files.
```python
>>> c = Compiler()
>>> c.load('docs/syndicate-protocols-schema-bundle.bin')
>>> type(c.root)
<class 'preserves.schema.Namespace'>
```
Attributes:
root (Namespace): the root namespace into which top-level schema modules are installed.
"""
def __init__(self):
self.root = Namespace(())
def load_filelike(self, f, module_name=None):
"""Reads a `meta.Bundle` or `meta.Schema` from the filelike object `f`, compiling and
installing it in `self.root`. If `f` contains a bundle, `module_name` is not used,
since the schema modules in the bundle know their own names; if `f` contains a plain
schema module, however, `module_name` is used directly if it is a string, and if it is
`None`, a suitable module name is computed from the `name` attribute of `f`, if it is
present. If `name` is absent in that case, `ValueError` is raised.
"""
x = Decoder(f.read()).next()
if x.key == SCHEMA:
if module_name is None:
if hasattr(f, 'name'):
module_name = pathlib.Path(f.name).stem
else:
raise ValueError('Cannot load schema module from filelike object without a module_name')
self.load_schema((Symbol(module_name),), x)
elif x.key == BUNDLE:
for (p, s) in x[0].items():
self.load_schema(p, s)
def load(self, filename):
"""Opens the file at `filename`, passing the resulting file object to
[load_filelike][preserves.schema.Compiler.load_filelike]."""
filename = pathlib.Path(filename)
with open(filename, 'rb') as f:
x = Decoder(f.read()).next()
if x.key == SCHEMA:
self.load_schema((Symbol(filename.stem),), x)
elif x.key == BUNDLE:
for (p, s) in x[0].items():
self.load_schema(p, s)
self.load_filelike(f, filename.stem)
def load_schema(self, module_path, schema):
if schema[0][VERSION] != 1:
@ -484,12 +958,51 @@ class Compiler:
ns[n] = c
def load_schema_file(filename):
"""Simple entry point to the compiler: creates a [Compiler][preserves.schema.Compiler],
calls [load][preserves.schema.Compiler.load] on it, and returns its `root`
[Namespace][preserves.schema.Namespace].
```python
>>> bundle = load_schema_file('docs/syndicate-protocols-schema-bundle.bin')
>>> type(bundle)
<class 'preserves.schema.Namespace'>
```
"""
c = Compiler()
c.load(filename)
return c.root
# a decorator
def extend(cls):
"""A decorator for function definitions. Useful for adding *behaviour* to the classes
resulting from loading a schema module:
```python
>>> bundle = load_schema_file('docs/syndicate-protocols-schema-bundle.bin')
>>> @extend(bundle.stream.LineMode.lf)
... def what_am_i(self):
... return 'I am a LINEFEED linemode'
>>> @extend(bundle.stream.LineMode.crlf)
... def what_am_i(self):
... return 'I am a CARRIAGE-RETURN-PLUS-LINEFEED linemode'
>>> bundle.stream.LineMode.lf()
LineMode.lf()
>>> bundle.stream.LineMode.lf().what_am_i()
'I am a LINEFEED linemode'
>>> bundle.stream.LineMode.crlf()
LineMode.crlf()
>>> bundle.stream.LineMode.crlf().what_am_i()
'I am a CARRIAGE-RETURN-PLUS-LINEFEED linemode'
```
"""
@wraps(cls)
def extender(f):
setattr(cls, f.__name__, f)
return f
@ -497,6 +1010,8 @@ def extend(cls):
__metaschema_filename = pathlib.Path(__file__).parent / 'schema.prb'
meta = load_schema_file(__metaschema_filename).schema
"""Schema module [Namespace][preserves.schema.Namespace] corresponding to [Preserves Schema's
metaschema](https://preserves.dev/preserves-schema.html#appendix-metaschema)."""
if __name__ == '__main__':
with open(__metaschema_filename, 'rb') as f:

226
implementations/python/preserves/text.py
View File

@ -1,3 +1,20 @@
"""The [preserves.text][] module implements the [Preserves human-readable text
syntax](https://preserves.dev/preserves-text.html).
The main entry points are functions [stringify][preserves.text.stringify],
[parse][preserves.text.parse], and
[parse_with_annotations][preserves.text.parse_with_annotations].
```python
>>> stringify(Record(Symbol('hi'), [1, [2, 3]]))
'<hi 1 [2 3]>'
>>> parse('<hi 1 [2 3]>')
#hi(1, (2, 3))
```
"""
import numbers
import struct
import base64
@ -8,11 +25,110 @@ from .error import *
from .compat import basestring_, unichr_
from .binary import Decoder
class TextCodec(object): pass
class TextCodec(object):
pass
NUMBER_RE = re.compile(r'^([-+]?\d+)(((\.\d+([eE][-+]?\d+)?)|([eE][-+]?\d+))([fF]?))?$')
class Parser(TextCodec):
"""Parser for the human-readable Preserves text syntax.
Args:
input_buffer (str):
initial contents of the input buffer; may subsequently be extended by calling
[extend][preserves.text.Parser.extend].
include_annotations (bool):
if `True`, wrap each value and subvalue in an
[Annotated][preserves.values.Annotated] object.
parse_embedded:
function accepting a `Value` and returning a possibly-decoded form of that value
suitable for placing into an [Embedded][preserves.values.Embedded] object.
Normal usage is to supply input text, and keep calling [next][preserves.text.Parser.next]
until a [ShortPacket][preserves.error.ShortPacket] exception is raised:
```python
>>> d = Parser('123 "hello" @x []')
>>> d.next()
123
>>> d.next()
'hello'
>>> d.next()
()
>>> d.next()
Traceback (most recent call last):
...
preserves.error.ShortPacket: Short input buffer
```
Alternatively, keep calling [try_next][preserves.text.Parser.try_next] until it yields
`None`, which is not in the domain of Preserves `Value`s:
```python
>>> d = Parser('123 "hello" @x []')
>>> d.try_next()
123
>>> d.try_next()
'hello'
>>> d.try_next()
()
>>> d.try_next()
```
For convenience, [Parser][preserves.text.Parser] implements the iterator interface,
backing it with [try_next][preserves.text.Parser.try_next], so you can simply iterate
over all complete values in an input:
```python
>>> d = Parser('123 "hello" @x []')
>>> list(d)
[123, 'hello', ()]
```
```python
>>> for v in Parser('123 "hello" @x []'):
... print(repr(v))
123
'hello'
()
```
Supply `include_annotations=True` to read annotations alongside the annotated values:
```python
>>> d = Parser('123 "hello" @x []', include_annotations=True)
>>> list(d)
[123, 'hello', @#x ()]
```
If you are incrementally reading from, say, a socket, you can use
[extend][preserves.text.Parser.extend] to add new input as if comes available:
```python
>>> d = Parser('123 "he')
>>> d.try_next()
123
>>> d.try_next() # returns None because the input is incomplete
>>> d.extend('llo"')
>>> d.try_next()
'hello'
>>> d.try_next()
```
Attributes:
input_buffer (str): buffered input waiting to be processed
index (int): read position within `input_buffer`
"""
def __init__(self, input_buffer=u'', include_annotations=False, parse_embedded=lambda x: x):
super(Parser, self).__init__()
self.input_buffer = input_buffer
@ -21,6 +137,8 @@ class Parser(TextCodec):
self.parse_embedded = parse_embedded
def extend(self, text):
"""Appends `text` to the remaining contents of `self.input_buffer`, trimming already-processed
text from the front of `self.input_buffer` and resetting `self.index` to zero."""
self.input_buffer = self.input_buffer[self.index:] + text
self.index = 0
@ -200,6 +318,11 @@ class Parser(TextCodec):
return Annotated(v) if self.include_annotations else v
def next(self):
"""Reads the next complete `Value` from the internal buffer, raising
[ShortPacket][preserves.error.ShortPacket] if too few bytes are available, or
[DecodeError][preserves.error.DecodeError] if the input is invalid somehow.
"""
self.skip_whitespace()
c = self.peek()
if c == '"':
@ -264,6 +387,8 @@ class Parser(TextCodec):
return self.wrap(self.read_raw_symbol_or_number([c]))
def try_next(self):
"""Like [next][preserves.text.Parser.next], but returns `None` instead of raising
[ShortPacket][preserves.error.ShortPacket]."""
start = self.index
try:
return self.next()
@ -280,25 +405,88 @@ class Parser(TextCodec):
raise StopIteration
return v
def parse(bs, **kwargs):
return Parser(input_buffer=bs, **kwargs).next()
def parse(text, **kwargs):
"""Yields the first complete encoded value from `text`, passing `kwargs` through to the
[Parser][preserves.text.Parser] constructor. Raises exceptions as per
[next][preserves.text.Parser.next].
Args:
text (str): encoded data to decode
"""
return Parser(input_buffer=text, **kwargs).next()
def parse_with_annotations(bs, **kwargs):
"""Like [parse][preserves.text.parse], but supplying `include_annotations=True` to the
[Parser][preserves.text.Parser] constructor."""
return Parser(input_buffer=bs, include_annotations=True, **kwargs).next()
class Formatter(TextCodec):
"""Printer (and indenting pretty-printer) for producing human-readable syntax from
Preserves `Value`s.
```python
>>> f = Formatter()
>>> f.append({'a': 1, 'b': 2})
>>> f.append(Record(Symbol('label'), ['field1', ['field2item1', 'field2item2']]))
>>> print(f.contents())
{"a": 1 "b": 2} <label "field1" ["field2item1" "field2item2"]>
>>> f = Formatter(indent=4)
>>> f.append({'a': 1, 'b': 2})
>>> f.append(Record(Symbol('label'), ['field1', ['field2item1', 'field2item2']]))
>>> print(f.contents())
{
"a": 1
"b": 2
}
<label "field1" [
"field2item1"
"field2item2"
]>
```
Args:
format_embedded:
function accepting an [Embedded][preserves.values.Embedded].embeddedValue and
returning a `Value` for serialization.
indent (int | None):
`None` disables indented pretty-printing; otherwise, an `int` specifies indentation
per nesting-level.
with_commas (bool):
`True` causes commas to separate sequence and set items and dictionary entries;
`False` omits commas.
trailing_comma (bool):
`True` causes a comma to be printed *after* the final item or entry in a sequence,
set or dictionary; `False` omits this trailing comma
include_annotations (bool):
`True` causes annotations to be included in the output; `False` causes them to be
omitted.
Attributes:
indent_delta (int): indentation per nesting-level
chunks (list[str]): fragments of output
"""
def __init__(self,
format_embedded=lambda x: x,
indent=None,
with_commas=False,
trailing_comma=False):
trailing_comma=False,
include_annotations=True):
super(Formatter, self).__init__()
self.indent_delta = 0 if indent is None else indent
self.indent_distance = 0
self.nesting = 0
self.with_commas = with_commas
self.trailing_comma = trailing_comma
self.chunks = []
self._format_embedded = format_embedded
self.include_annotations = include_annotations
def format_embedded(self, v):
if self._format_embedded is None:
@ -306,9 +494,12 @@ class Formatter(TextCodec):
return self._format_embedded(v)
def contents(self):
"""Returns a `str` constructed from the join of the chunks in `self.chunks`."""
return u''.join(self.chunks)
def is_indenting(self):
"""Returns `True` iff this [Formatter][preserves.text.Formatter] is in pretty-printing
indenting mode."""
return self.indent_delta > 0
def write_indent(self):
@ -352,6 +543,17 @@ class Formatter(TextCodec):
self.chunks.append(closer)
def append(self, v):
"""Extend `self.chunks` with at least one chunk, together making up the text
representation of `v`."""
if self.chunks and self.nesting == 0:
self.write_indent_space()
try:
self.nesting += 1
self._append(v)
finally:
self.nesting -= 1
def _append(self, v):
v = preserve(v)
if hasattr(v, '__preserve_write_text__'):
v.__preserve_write_text__(self)
@ -375,18 +577,18 @@ class Formatter(TextCodec):
else: self.write_stringlike_char(c)
self.chunks.append('"')
elif isinstance(v, list):
self.write_seq('[', ']', v, self.append)
self.write_seq('[', ']', v, self._append)
elif isinstance(v, tuple):
self.write_seq('[', ']', v, self.append)
self.write_seq('[', ']', v, self._append)
elif isinstance(v, set):
self.write_seq('#{', '}', v, self.append)
self.write_seq('#{', '}', v, self._append)
elif isinstance(v, frozenset):
self.write_seq('#{', '}', v, self.append)
self.write_seq('#{', '}', v, self._append)
elif isinstance(v, dict):
def append_kv(kv):
self.append(kv[0])
self._append(kv[0])
self.chunks.append(': ')
self.append(kv[1])
self._append(kv[1])
self.write_seq('{', '}', v.items(), append_kv)
else:
try:
@ -396,12 +598,14 @@ class Formatter(TextCodec):
if i is None:
self.cannot_format(v)
else:
self.write_seq('[', ']', i, self.append)
self.write_seq('[', ']', i, self._append)
def cannot_format(self, v):
raise TypeError('Cannot preserves-format: ' + repr(v))
def stringify(v, **kwargs):
"""Convert a single `Value` `v` to a string. Any supplied `kwargs` are passed on to the
underlying [Formatter][preserves.text.Formatter] constructor."""
e = Formatter(**kwargs)
e.append(v)
return e.contents()

405
implementations/python/preserves/values.py
View File

@ -1,3 +1,8 @@
"""The [preserves.values][] module implements the core representations of Preserves
[`Value`s](https://preserves.dev/preserves.html#semantics) as Python values.
"""
import re
import sys
import struct
@ -6,6 +11,16 @@ import math
from .error import DecodeError
def preserve(v):
"""Converts `v` to a representation of a Preserves `Value` by (repeatedly) setting
```python
v = v.__preserve__()
```
while `v` has a `__preserve__` method. Parsed [Schema][preserves.schema]
values are able to render themselves to their serialized representations this way.
"""
while hasattr(v, '__preserve__'):
v = v.__preserve__()
return v
@ -14,6 +29,10 @@ def float_to_int(v):
return struct.unpack('>Q', struct.pack('>d', v))[0]
def cmp_floats(a, b):
"""Implements the `totalOrder` predicate defined in section 5.10 of [IEEE Std
754-2008](https://dx.doi.org/10.1109/IEEESTD.2008.4610935).
"""
a = float_to_int(a)
b = float_to_int(b)
if a & 0x8000000000000000: a = a ^ 0x7fffffffffffffff
@ -21,6 +40,32 @@ def cmp_floats(a, b):
return a - b
class Float(object):
"""Wrapper for treating a Python double-precision floating-point value as a
single-precision (32-bit) float, from Preserves' perspective. (Python lacks native
single-precision floating point support.)
```python
>>> Float(3.45)
Float(3.45)
>>> import preserves
>>> preserves.stringify(Float(3.45))
'3.45f'
>>> preserves.stringify(3.45)
'3.45'
>>> preserves.parse('3.45f')
Float(3.45)
>>> preserves.parse('3.45')
3.45
>>> preserves.encode(Float(3.45))
b'\\x82@\\\\\\xcc\\xcd'
>>> preserves.encode(3.45)
b'\\x83@\\x0b\\x99\\x99\\x99\\x99\\x99\\x9a'
```
Attributes:
value (float): the double-precision representation of intended single-precision value
"""
def __init__(self, value):
self.value = value
@ -43,7 +88,30 @@ class Float(object):
def __repr__(self):
return 'Float(' + repr(self.value) + ')'
def _to_bytes(self):
def to_bytes(self):
"""Converts this 32-bit single-precision floating point value to its binary32 format,
taking care to preserve the quiet/signalling bit-pattern of NaN values, unlike its
`struct.pack('>f', ...)` equivalent.
```python
>>> Float.from_bytes(b'\\x7f\\x80\\x00{')
Float(nan)
>>> Float.from_bytes(b'\\x7f\\x80\\x00{').to_bytes()
b'\\x7f\\x80\\x00{'
>>> struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]
nan
>>> Float(struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]).to_bytes()
b'\\x7f\\xc0\\x00{'
>>> struct.pack('>f', struct.unpack('>f', b'\\x7f\\x80\\x00{')[0])
b'\\x7f\\xc0\\x00{'
```
(Note well the difference between `7f80007b` and `7fc0007b`!)
"""
if math.isnan(self.value) or math.isinf(self.value):
dbs = struct.pack('>d', self.value)
vd = struct.unpack('>Q', dbs)[0]
@ -56,16 +124,39 @@ class Float(object):
def __preserve_write_binary__(self, encoder):
encoder.buffer.append(0x82)
encoder.buffer.extend(self._to_bytes())
encoder.buffer.extend(self.to_bytes())
def __preserve_write_text__(self, formatter):
if math.isnan(self.value) or math.isinf(self.value):
formatter.chunks.append('#xf"' + self._to_bytes().hex() + '"')
formatter.chunks.append('#xf"' + self.to_bytes().hex() + '"')
else:
formatter.chunks.append(repr(self.value) + 'f')
@staticmethod
def from_bytes(bs):
"""Converts a 4-byte-long byte string to a 32-bit single-precision floating point value
wrapped in a [Float][preserves.values.Float] instance. Takes care to preserve the
quiet/signalling bit-pattern of NaN values, unlike its `struct.unpack('>f', ...)`
equivalent.
```python
>>> Float.from_bytes(b'\\x7f\\x80\\x00{')
Float(nan)
>>> Float.from_bytes(b'\\x7f\\x80\\x00{').to_bytes()
b'\\x7f\\x80\\x00{'
>>> struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]
nan
>>> Float(struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]).to_bytes()
b'\\x7f\\xc0\\x00{'
>>> struct.pack('>f', struct.unpack('>f', b'\\x7f\\x80\\x00{')[0])
b'\\x7f\\xc0\\x00{'
```
(Note well the difference between `7f80007b` and `7fc0007b`!)
"""
vf = struct.unpack('>I', bs)[0]
if (vf & 0x7f800000) == 0x7f800000:
# NaN or inf. Preserve quiet/signalling bit by manually expanding to double-precision.
@ -79,7 +170,35 @@ class Float(object):
# FIXME: This regular expression is conservatively correct, but Anglo-chauvinistic.
RAW_SYMBOL_RE = re.compile(r'^[-a-zA-Z0-9~!$%^&*?_=+/.]+$')
def _eq(a, b):
from .compare import eq
return eq(a, b)
class Symbol(object):
"""Representation of Preserves `Symbol`s.
```python
>>> Symbol('xyz')
#xyz
>>> Symbol('xyz').name
'xyz'
>>> import preserves
>>> preserves.stringify(Symbol('xyz'))
'xyz'
>>> preserves.stringify(Symbol('hello world'))
'|hello world|'
>>> preserves.parse('xyz')
#xyz
>>> preserves.parse('|hello world|')
#hello world
```
Attributes:
name (str | Symbol):
The symbol's text label. If an existing [Symbol][preserves.values.Symbol] is passed
in, the existing Symbol's `name` is used as the `name` for the new Symbol.
"""
def __init__(self, name):
self.name = name.name if isinstance(name, Symbol) else name
@ -125,6 +244,32 @@ class Symbol(object):
formatter.chunks.append('|')
class Record(object):
"""Representation of Preserves `Record`s, which are a pair of a *label* `Value` and a sequence of *field* `Value`s.
```python
>>> r = Record(Symbol('label'), ['field1', ['field2item1', 'field2item2']])
>>> r
#label('field1', ['field2item1', 'field2item2'])
>>> r.key
#label
>>> r.fields
('field1', ['field2item1', 'field2item2'])
>>> import preserves
>>> preserves.stringify(r)
'<label "field1" ["field2item1" "field2item2"]>'
>>> r == preserves.parse('<label "field1" ["field2item1" "field2item2"]>')
True
```
Args:
key (Value): the `Record`'s label
fields (iterable[Value]): the fields of the `Record`
Attributes:
key (Value): the `Record`'s label
fields (tuple[Value]): the fields of the `Record`
"""
def __init__(self, key, fields):
self.key = key
self.fields = tuple(fields)
@ -132,7 +277,7 @@ class Record(object):
def __eq__(self, other):
other = _unwrap(other)
return isinstance(other, Record) and (self.key, self.fields) == (other.key, other.fields)
return isinstance(other, Record) and _eq((self.key, self.fields), (other.key, other.fields))
def __ne__(self, other):
return not self.__eq__(other)
@ -165,10 +310,69 @@ class Record(object):
@staticmethod
def makeConstructor(labelSymbolText, fieldNames):
"""
Equivalent to `Record.makeBasicConstructor(Symbol(labelSymbolText), fieldNames)`.
Deprecated:
Use [preserves.schema][] definitions instead.
"""
return Record.makeBasicConstructor(Symbol(labelSymbolText), fieldNames)
@staticmethod
def makeBasicConstructor(label, fieldNames):
"""Constructs and returns a "constructor" for `Record`s having a certain `label` and
number of fields.
Deprecated:
Use [preserves.schema][] definitions instead.
The "constructor" is a callable function that accepts `len(fields)` arguments and
returns a [Record][preserves.values.Record] with `label` as its label and the arguments
to the constructor as field values.
In addition, the "constructor" has a `constructorInfo` attribute holding a
[RecordConstructorInfo][preserves.values.RecordConstructorInfo] object, an `isClassOf`
attribute holding a unary function that returns `True` iff its argument is a
[Record][preserves.values.Record] with label `label` and arity `len(fieldNames)`, and
an `ensureClassOf` attribute that raises an `Exception` if `isClassOf` returns false on
its argument and returns the argument otherwise.
Finally, for each field name `f` in `fieldNames`, the "constructor" object has an
attribute `_f` that is a unary function that retrieves the `f` field from the passed in
argument.
```python
>>> c = Record.makeBasicConstructor(Symbol('date'), 'year month day')
>>> c(1969, 7, 16)
#date(1969, 7, 16)
>>> c.constructorInfo
#date/3
>>> c.isClassOf(c(1969, 7, 16))
True
>>> c.isClassOf(Record(Symbol('date'), [1969, 7, 16]))
True
>>> c.isClassOf(Record(Symbol('date'), [1969]))
False
>>> c.ensureClassOf(c(1969, 7, 16))
#date(1969, 7, 16)
>>> c.ensureClassOf(Record(Symbol('date'), [1969]))
Traceback (most recent call last):
...
TypeError: Record: expected #date/3, got #date(1969)
>>> c._year(c(1969, 7, 16))
1969
>>> c._month(c(1969, 7, 16))
7
>>> c._day(c(1969, 7, 16))
16
```
Args:
label (Value): Label to use for constructed/matched `Record`s
fieldNames (tuple[str] | list[str] | str): Names of the `Record`'s fields
"""
if type(fieldNames) == str:
fieldNames = fieldNames.split()
arity = len(fieldNames)
@ -196,6 +400,19 @@ class Record(object):
return ctor
class RecordConstructorInfo(object):
"""Describes the shape of a `Record` constructor, namely its *label* and its *arity* (field
count).
```python
>>> RecordConstructorInfo(Symbol('label'), 3)
#label/3
```
Attributes:
key (Value): the label of matching `Record`s
arity (int): the number of fields in matching `Record`s
"""
def __init__(self, key, arity):
self.key = key
self.arity = arity
@ -203,7 +420,7 @@ class RecordConstructorInfo(object):
def __eq__(self, other):
other = _unwrap(other)
return isinstance(other, RecordConstructorInfo) and \
(self.key, self.arity) == (other.key, other.arity)
_eq((self.key, self.arity), (other.key, other.arity))
def __ne__(self, other):
return not self.__eq__(other)
@ -218,6 +435,29 @@ class RecordConstructorInfo(object):
# Blub blub blub
class ImmutableDict(dict):
"""A subclass of Python's built-in `dict` that overrides methods that could mutate the
dictionary, causing them to raise `TypeError('Immutable')` if called.
Implements the `__hash__` method, allowing [ImmutableDict][preserves.values.ImmutableDict]
instances to be used whereever immutable data are permitted; in particular, as keys in
other dictionaries.
```python
>>> d = ImmutableDict([('a', 1), ('b', 2)])
>>> d
{'a': 1, 'b': 2}
>>> d['c'] = 3
Traceback (most recent call last):
...
TypeError: Immutable
>>> del d['b']
Traceback (most recent call last):
...
TypeError: Immutable
```
"""
def __init__(self, *args, **kwargs):
if hasattr(self, '__hash'): raise TypeError('Immutable')
super(ImmutableDict, self).__init__(*args, **kwargs)
@ -241,6 +481,23 @@ class ImmutableDict(dict):
@staticmethod
def from_kvs(kvs):
"""Constructs an [ImmutableDict][preserves.values.ImmutableDict] from a sequence of
alternating keys and values; compare to the
[ImmutableDict][preserves.values.ImmutableDict] constructor, which takes a sequence of
key-value pairs.
```python
>>> ImmutableDict.from_kvs(['a', 1, 'b', 2])
{'a': 1, 'b': 2}
>>> ImmutableDict.from_kvs(['a', 1, 'b', 2])['c'] = 3
Traceback (most recent call last):
...
TypeError: Immutable
```
"""
i = iter(kvs)
result = ImmutableDict()
result_proxy = super(ImmutableDict, result)
@ -257,6 +514,15 @@ class ImmutableDict(dict):
return result
def dict_kvs(d):
"""Generator function yielding a sequence of alternating keys and values from `d`. In some
sense the inverse of [ImmutableDict.from_kvs][preserves.values.ImmutableDict.from_kvs].
```python
>>> list(dict_kvs({'a': 1, 'b': 2}))
['a', 1, 'b', 2]
```
"""
for k in d:
yield k
yield d[k]
@ -264,31 +530,77 @@ def dict_kvs(d):
inf = float('inf')
class Annotated(object):
"""A Preserves `Value` along with a sequence of `Value`s *annotating* it. Compares equal to
the underlying `Value`, ignoring the annotations. See the [specification document for more
about annotations](https://preserves.dev/preserves-text.html#annotations).
```python
>>> import preserves
>>> a = preserves.parse('''
... ; A comment
... [1 2 3]
... ''', include_annotations=True)
>>> a
@' A comment' (1, 2, 3)
>>> a.item
(1, 2, 3)
>>> a.annotations
[' A comment']
>>> a == (1, 2, 3)
True
>>> a == preserves.parse('@xyz [1 2 3]', include_annotations=True)
True
>>> a[0]
Traceback (most recent call last):
...
TypeError: 'Annotated' object is not subscriptable
>>> a.item[0]
1
>>> type(a.item[0])
<class 'preserves.values.Annotated'>
>>> a.item[0].annotations
[]
>>> print(preserves.stringify(a))
@" A comment" [1 2 3]
>>> print(preserves.stringify(a, include_annotations=False))
[1 2 3]
```
Attributes:
item (Value): the underlying annotated `Value`
annotations (list[Value]): the annotations attached to `self.item`
"""
def __init__(self, item):
self.annotations = []
self.item = item
def __preserve_write_binary__(self, encoder):
for a in self.annotations:
encoder.buffer.append(0x85)
encoder.append(a)
if encoder.include_annotations:
for a in self.annotations:
encoder.buffer.append(0x85)
encoder.append(a)
encoder.append(self.item)
def __preserve_write_text__(self, formatter):
for a in self.annotations:
formatter.chunks.append('@')
formatter.append(a)
formatter.chunks.append(' ')
if formatter.include_annotations:
for a in self.annotations:
formatter.chunks.append('@')
formatter.append(a)
formatter.chunks.append(' ')
formatter.append(self.item)
def strip(self, depth=inf):
"""Calls [strip_annotations][preserves.values.strip_annotations] on `self` and `depth`."""
return strip_annotations(self, depth)
def peel(self):
"""Calls [strip_annotations][preserves.values.strip_annotations] on `self` with `depth=1`."""
return strip_annotations(self, 1)
def __eq__(self, other):
return self.item == _unwrap(other)
return _eq(self.item, _unwrap(other))
def __ne__(self, other):
return not self.__eq__(other)
@ -300,9 +612,30 @@ class Annotated(object):
return ' '.join(list('@' + repr(a) for a in self.annotations) + [repr(self.item)])
def is_annotated(v):
"""`True` iff `v` is an instance of [Annotated][preserves.values.Annotated]."""
return isinstance(v, Annotated)
def strip_annotations(v, depth=inf):
"""Exposes `depth` layers of raw structure of
potentially-[Annotated][preserves.values.Annotated] `Value`s. If `depth==0` or `v` is not
[Annotated][preserves.values.Annotated], just returns `v`. Otherwise, descends recursively
into the structure of `v.item`.
```python
>>> import preserves
>>> a = preserves.parse('@"A comment" [@a 1 @b 2 @c 3]', include_annotations=True)
>>> is_annotated(a)
True
>>> print(preserves.stringify(a))
@"A comment" [@a 1 @b 2 @c 3]
>>> print(preserves.stringify(strip_annotations(a)))
[1 2 3]
>>> print(preserves.stringify(strip_annotations(a, depth=1)))
[@a 1 @b 2 @c 3]
```
"""
if depth == 0: return v
if not is_annotated(v): return v
@ -329,6 +662,18 @@ def strip_annotations(v, depth=inf):
return v
def annotate(v, *anns):
"""Wraps `v` in an [Annotated][preserves.values.Annotated] object, if it isn't already
wrapped, and appends each of the `anns` to the [Annotated][preserves.values.Annotated]'s
`annotations` sequence. NOTE: Does not recursively ensure that any parts of the argument
`v` are themselves wrapped in [Annotated][preserves.values.Annotated] objects!
```python
>>> import preserves
>>> print(preserves.stringify(annotate(123, "A comment", "Another comment")))
@"A comment" @"Another comment" 123
```
"""
if not is_annotated(v):
v = Annotated(v)
for a in anns:
@ -342,8 +687,38 @@ def _unwrap(x):
return x
class Embedded:
def __init__(self, value):
self.embeddedValue = value
"""Representation of a Preserves `Embedded` value. For more on the meaning and use of
embedded values, [see the specification](https://preserves.dev/preserves.html#embeddeds).
```python
>>> import io
>>> e = Embedded(io.StringIO('some text'))
>>> e # doctest: +ELLIPSIS
#!<_io.StringIO object at ...>
>>> e.embeddedValue # doctest: +ELLIPSIS
<_io.StringIO object at ...>
```
```python
>>> import preserves
>>> print(preserves.stringify(Embedded(None)))
Traceback (most recent call last):
...
TypeError: Cannot preserves-format: None
>>> print(preserves.stringify(Embedded(None), format_embedded=lambda x: 'abcdef'))
#!"abcdef"
```
Attributes:
embeddedValue:
any Python value; could be a platform object, could be a representation of a
Preserves `Value`, could be `None`, could be anything!
"""
def __init__(self, embeddedValue):
self.embeddedValue = embeddedValue
def __eq__(self, other):
other = _unwrap(other)

6
implementations/python/setup.py
View File

@ -2,7 +2,7 @@ from setuptools import setup
setup(
name="preserves",
version="0.16.1",
version="0.18.1",
author="Tony Garnock-Jones",
author_email="tonyg@leastfixedpoint.com",
license="Apache Software License",
@ -14,8 +14,10 @@ setup(
"Programming Language :: Python :: 3",
],
packages=["preserves"],
url="https://gitlab.com/preserves/preserves",
url="https://preserves.dev/",
description="Experimental data serialization format",
long_description=open("README.md").read(),
long_description_content_type="text/markdown",
install_requires=[],
python_requires=">=3.6, <4",
setup_requires=['setuptools_scm'],

17
implementations/python/tests/test_doctests.py Normal file
View File

@ -0,0 +1,17 @@
import doctest
import pkgutil
import importlib
import preserves
def load_tests(loader, tests, ignore):
mods = []
mods.append(preserves)
for mi in pkgutil.iter_modules(preserves.__path__, preserves.__name__ + '.'):
mod = importlib.import_module(mi.name)
mods.append(mod)
for mod in mods:
tests.addTests(doctest.DocTestSuite(mod))
return tests

8
implementations/python/tests/test_preserves.py
View File

@ -261,7 +261,7 @@ def install_test(d, variant, tName, binaryForm, annotatedTextForm):
def test_back(self): self.assertPreservesEqual(self.DS(binaryForm), back)
def test_back_ann(self): self.assertPreservesEqual(self.D(self.E(annotatedTextForm)), annotatedTextForm)
def test_encode(self): self.assertPreservesEqual(self.E(forward), binaryForm)
def test_encode_canonical(self): self.assertPreservesEqual(self.EC(annotatedTextForm), binaryForm)
def test_encode_nondet(self): self.assertPreservesEqual(self.ENONDET(annotatedTextForm), binaryForm)
def test_encode_ann(self): self.assertPreservesEqual(self.E(annotatedTextForm), binaryForm)
add_method(d, tName, test_match_expected)
add_method(d, tName, test_roundtrip)
@ -271,7 +271,7 @@ def install_test(d, variant, tName, binaryForm, annotatedTextForm):
if variant in ['normal']:
add_method(d, tName, test_encode)
if variant in ['nondeterministic']:
add_method(d, tName, test_encode_canonical)
add_method(d, tName, test_encode_nondet)
if variant in ['normal', 'nondeterministic']:
add_method(d, tName, test_encode_ann)
@ -323,8 +323,8 @@ class CommonTestSuite(PreservesTestCase):
def E(self, v):
return encode(v, encode_embedded=lambda x: x)
def EC(self, v):
return encode(v, encode_embedded=lambda x: x, canonicalize=True)
def ENONDET(self, v):
return encode(v, encode_embedded=lambda x: x, canonicalize=True, include_annotations=True)
class RecordTests(PreservesTestCase):
def test_getters(self):

25
implementations/python/update-doc-versions.sh Executable file
View File

@ -0,0 +1,25 @@
#!/bin/sh
latestversion=$(git tag | fgrep python-preserves@ | cut -d@ -f2 | sort -V -r | head -1)
(
firstitem='y';
printf '[';
for version in $(ls ../../python/*/sitemap.xml | cut -d/ -f4 | grep -v 'latest' | grep -v 'dev' | sort -V -r)
do
if [ "$firstitem" = "y" ]
then
firstitem=n
else
printf ','
fi
if [ "$version" = "$latestversion" ]
then
aliases='["latest"]'
else
aliases='[]'
fi
printf '\n {"version":"%s","title":"%s","aliases":%s}' "$version" "$version" "$aliases"
done;
printf '\n]'
) | tee ../../_data/python-versions.json
rm -f ../../python/latest
ln -s "$latestversion" ../../python/latest

129
implementations/racket/preserves/preserves-schema/bin/preserves-schema-rkt.rkt
View File

@ -19,51 +19,38 @@
(define (clean-input p)
(path->string (simplify-path (path->complete-path (expand-user-path p)) #f)))
(define (compute-base paths)
(match paths
['() "."]
[(list p) (path->string (simplify-path (build-path p 'up) #f))]
[_ (let try-index ((i 0))
(let scan-paths ((paths paths) (ch #f))
(match paths
['() (try-index (+ i 1))]
[(cons p more-paths)
(cond [(= i (string-length p)) (substring p 0 i)]
[(not ch) (scan-paths more-paths (string-ref p i))]
[(eqv? ch (string-ref p i)) (scan-paths more-paths ch)]
[else (substring p 0 i)])])))]))
(define (expand-globs globs base0 output-directory0)
(for/list [(entry (in-list (remove-duplicates
#:key car
(append-map (lambda (g)
(define results (glob g))
(when (null? results)
(error 'preserves-schema-rkt
"Input not found: ~v" g))
(map (lambda (r) (list (path->string r) g))
results))
globs))))]
(match-define (list full-input-path generating-glob) entry)
(define base (or base0 (path->string (simplify-path (build-path generating-glob 'up) #f))))
(define output-directory (or (and output-directory0 (clean-input output-directory0)) base))
(define (expand-globs globs base0 output-directory0 k)
(define base (or base0 (compute-base globs)))
(define output-directory (or (and output-directory0 (clean-input output-directory0))
base))
(k base
output-directory
(for/list [(full-input-path (in-list (map path->string
(remove-duplicates
(append-map (lambda (g)
(define results (glob g))
(when (null? results)
(error 'preserves-schema-rkt
"Input not found: ~v" g))
results)
globs)))))]
(when (not (string-prefix? full-input-path base))
(error 'preserves-schema-rkt "Input filename ~v falls outside base ~v"
full-input-path
base))
(define relative-input-path (substring full-input-path (string-length base)))
(define module-path (for/list [(p (explode-path relative-input-path))]
(string->symbol (path->string (path-replace-extension p "")))))
(define relative-output-path
(path->string (path-replace-extension relative-input-path ".rkt")))
(log-info "Loading Preserves Schema file ~s" full-input-path)
(schema (schema-translation-paths full-input-path
relative-input-path
(path->string (build-path output-directory relative-output-path))
relative-output-path)
module-path
(file->schema full-input-path)))))
(when (not (string-prefix? full-input-path base))
(error 'preserves-schema-rkt "Input filename ~v falls outside base ~v"
full-input-path
base))
(define relative-input-path (substring full-input-path (string-length base)))
(define module-path (for/list [(p (explode-path relative-input-path))]
(string->symbol (path->string (path-replace-extension p "")))))
(define relative-output-path
(path->string (path-replace-extension relative-input-path ".rkt")))
(log-info "Loading Preserves Schema file ~s" full-input-path)
(schema (schema-translation-paths full-input-path
relative-input-path
(path->string (build-path output-directory relative-output-path))
relative-output-path)
module-path
(file->schema full-input-path))))
(define (batch-compile #:inputs inputs
#:additional-modules [additional-modules (hash)]
@ -71,39 +58,33 @@
#:output-directory [output-directory #f]
#:write-files? [write-files? #t]
#:plugins [plugins '()])
(expand-globs inputs
base-directory
output-directory
(lambda (base-directory output-directory schemas)
(define schemas (expand-globs inputs base-directory output-directory))
(define index
(for/fold [(index additional-modules)]
[(s (in-list schemas))]
(hash-set index
(schema-module-path s)
(schema-translation-paths-relative-output-path (schema-paths s)))))
(define index
(for/fold [(index additional-modules)]
[(s (in-list schemas))]
(hash-set index
(schema-module-path s)
(schema-translation-paths-relative-output-path (schema-paths s)))))
(define outputs
(for/hash [(s (in-list schemas))]
(match-define (schema (and tps (schema-translation-paths fi ri fo ro))
mp
v)
s)
(values fo
(schema->module-stx
(last mp)
(lambda (module-path) (hash-ref index module-path #f))
v
#:translation-paths tps
#:plugins plugins))))
(define outputs
(for/hash [(s (in-list schemas))]
(match-define (schema (and tps (schema-translation-paths fi ri fo ro)) mp v) s)
(values fo
(schema->module-stx
(last mp)
(lambda (module-path) (hash-ref index module-path #f))
v
#:translation-paths tps
#:plugins plugins))))
(when write-files?
(for [((output-path stx) (in-hash outputs))]
(make-parent-directory* output-path)
(with-output-to-file output-path #:exists 'replace
(lambda () (pretty-write stx)))))
(when write-files?
(for [((output-path stx) (in-hash outputs))]
(make-parent-directory* output-path)
(with-output-to-file output-path #:exists 'replace
(lambda () (pretty-write stx)))))
outputs)))
outputs)
(define (load-plugins mods)
(for/list [(mod mods)]
@ -136,6 +117,8 @@
(error '--module "Argument must be Namespace=path: ~v" namespace=path))
(let* ((namespace-str (substring namespace=path 0 i))
(path-str (substring namespace=path (+ i 1))))
(when (string-prefix? path-str ":")
(set! path-str (string->symbol (substring path-str 1))))
(set! additional-modules
(cons (list (map string->symbol (string-split namespace-str "."))
path-str)

22
implementations/racket/preserves/preserves-schema/gen/schema.rkt
View File

@ -370,16 +370,7 @@
(define parse-DictionaryEntries!
(parse-success-or-error 'parse-DictionaryEntries parse-DictionaryEntries))
(define (EmbeddedTypeName? p)
(or (EmbeddedTypeName-Ref? p) (EmbeddedTypeName-false? p)))
(struct
EmbeddedTypeName-Ref
(value)
#:transparent
#:methods
gen:preservable
((define/generic *->preserve ->preserve)
(define (->preserve preservable)
(match preservable ((EmbeddedTypeName-Ref src) (*->preserve src))))))
(or (EmbeddedTypeName-false? p) (EmbeddedTypeName-Ref? p)))
(struct
EmbeddedTypeName-false
()
@ -389,11 +380,20 @@
((define/generic *->preserve ->preserve)
(define (->preserve preservable)
(match preservable ((EmbeddedTypeName-false) '#f)))))
(struct
EmbeddedTypeName-Ref
(value)
#:transparent
#:methods
gen:preservable
((define/generic *->preserve ->preserve)
(define (->preserve preservable)
(match preservable ((EmbeddedTypeName-Ref src) (*->preserve src))))))
(define (parse-EmbeddedTypeName input)
(match
input
((app parse-Ref (and dest (not (== eof)))) (EmbeddedTypeName-Ref dest))
((and dest (== '#f)) (EmbeddedTypeName-false))
((app parse-Ref (and dest (not (== eof)))) (EmbeddedTypeName-Ref dest))
(_ eof)))
(define parse-EmbeddedTypeName!
(parse-success-or-error 'parse-EmbeddedTypeName parse-EmbeddedTypeName))

2
implementations/racket/preserves/preserves-schema/schema.prs
View File

@ -17,7 +17,7 @@ Schema = <schema {
; version 1 .
Version = 1 .
EmbeddedTypeName = Ref / #f.
EmbeddedTypeName = #f / Ref .
Definitions = { symbol: Definition ...:... }.

2
implementations/rust/preserves-path/build.rs
View File

@ -12,7 +12,7 @@ fn main() -> Result<(), Error> {
let mut c = CompilerConfig::new(gen_dir, "crate::schemas".to_owned());
let inputs = expand_inputs(&vec!["path.bin".to_owned()])?;
c.load_schemas_and_bundles(&inputs)?;
c.load_schemas_and_bundles(&inputs, &vec![])?;
compile(&c)
}

7
implementations/rust/preserves-schema/src/bin/preserves-schema-rs.rs
View File

@ -19,6 +19,9 @@ struct CommandLine {
#[structopt(long)]
module: Vec<String>,
#[structopt(long)]
xref: Vec<String>,
input_glob: Vec<String>,
}
@ -40,6 +43,8 @@ fn main() -> Result<(), Error> {
if let Some(c) = args.support_crate {
config.support_crate = c;
}
config.load_schemas_and_bundles(&expand_inputs(&args.input_glob)?)?;
config.load_schemas_and_bundles(
&expand_inputs(&args.input_glob)?,
&expand_inputs(&args.xref)?)?;
compile(&config)
}

12
implementations/rust/preserves-schema/src/compiler/context.rs
View File

@ -16,6 +16,7 @@ use preserves::value::Value;
use super::CompilerConfig;
use super::names;
use super::types;
use super::types::Purpose;
pub struct BundleContext<'b> {
pub config: &'b CompilerConfig,
@ -82,8 +83,9 @@ impl<'b> BundleContext<'b> {
"_Value"
}
pub fn lookup_definition(&self, r: &Ref) -> Option<&Definition> {
self.config.bundle.get(&r.module.0).and_then(|s| s.definitions.0.get(&r.name))
pub fn lookup_definition(&self, r: &Ref) -> Option<(&Definition, Purpose)> {
self.config.bundle.get(&r.module.0).and_then(
|s| s.0.definitions.0.get(&r.name).map(|d| (d, s.1)))
}
pub fn type_for_name(&self, r: &Ref) -> Option<&types::TDefinition> {
@ -229,7 +231,7 @@ impl<'a, 'm, 'b> FunctionContext<'a, 'm, 'b> {
pub fn new(m: &'a mut ModuleContext<'m, 'b>, error_context: &str) -> Self {
FunctionContext {
error_context: error_context.to_owned(),
m: m,
m,
temp_counter: 0,
captures: Vec::new(),
capture_mode: CaptureMode::Definite,
@ -238,8 +240,8 @@ impl<'a, 'm, 'b> FunctionContext<'a, 'm, 'b> {
pub fn capture(&mut self, field_name: String, ty: types::TField, source_expr: String) {
self.captures.push(Capture {
field_name: field_name,
ty: ty,
field_name,
ty,
source_expr: match self.capture_mode {
CaptureMode::Definite =>
source_expr,

118
implementations/rust/preserves-schema/src/compiler/mod.rs
View File

@ -8,6 +8,7 @@ pub mod unparsers;
use crate::*;
use crate::compiler::context::*;
use crate::compiler::types::Purpose;
use crate::gen::Language;
use crate::gen::schema;
use crate::gen::schema::*;
@ -16,7 +17,7 @@ use crate::syntax::block::constructors::*;
use glob::glob;
use preserves::value::BinarySource;
use preserves::value::IOBinarySource;
use preserves::value::BytesBinarySource;
use preserves::value::Map;
use preserves::value::Set;
use preserves::value::Reader;
@ -97,7 +98,7 @@ impl ExternalModule {
#[derive(Debug)]
pub struct CompilerConfig {
pub bundle: Map<ModulePath, Schema>,
pub bundle: Map<ModulePath, (Schema, Purpose)>,
pub output_dir: PathBuf,
pub fully_qualified_module_prefix: String,
pub support_crate: String,
@ -105,20 +106,59 @@ pub struct CompilerConfig {
pub plugins: Vec<Box<dyn Plugin>>,
}
pub fn load_schema_or_bundle(bundle: &mut Map<ModulePath, Schema>, i: &PathBuf) -> io::Result<()> {
pub fn load_schema_or_bundle_with_purpose(
bundle: &mut Map<ModulePath, (Schema, Purpose)>,
i: &PathBuf,
purpose: Purpose,
) -> io::Result<()> {
let mut inserted = Map::<ModulePath, Schema>::new();
load_schema_or_bundle(&mut inserted, i)?;
for (k, v) in inserted.into_iter() { bundle.insert(k, (v, purpose)); }
Ok(())
}
pub fn load_schema_or_bundle_bin_with_purpose(
bundle: &mut Map<ModulePath, (Schema, Purpose)>,
prefix: &str,
input: &[u8],
purpose: Purpose,
) -> io::Result<()> {
let mut inserted = Map::<ModulePath, Schema>::new();
load_schema_or_bundle_bin(&mut inserted, prefix, input)?;
for (k, v) in inserted.into_iter() { bundle.insert(k, (v, purpose)); }
Ok(())
}
fn bundle_prefix(i: &PathBuf) -> io::Result<&str> {
i.file_stem().ok_or_else(
|| io::Error::new(io::ErrorKind::InvalidData,
format!("Bad schema file stem: {:?}", i)))?
.to_str().ok_or_else(
|| io::Error::new(io::ErrorKind::InvalidData,
format!("Invalid UTF-8 in schema file name: {:?}", i)))
}
pub fn load_schema_or_bundle(
bundle: &mut Map<ModulePath, Schema>,
i: &PathBuf,
) -> io::Result<()> {
let mut f = File::open(&i)?;
let mut src = IOBinarySource::new(&mut f);
let mut reader = src.packed();
let blob = reader.next_iovalue(false)?;
let mut bs = vec![];
f.read_to_end(&mut bs)?;
load_schema_or_bundle_bin(bundle, bundle_prefix(i)?, &bs[..])
}
pub fn load_schema_or_bundle_bin(
bundle: &mut Map<ModulePath, Schema>,
prefix: &str,
input: &[u8],
) -> io::Result<()> {
let mut src = BytesBinarySource::new(input);
let mut reader = src.packed_iovalues();
let blob = reader.demand_next(false)?;
let language = Language::default();
if let Ok(s) = language.parse(&blob) {
let prefix = i.file_stem().ok_or_else(
|| io::Error::new(io::ErrorKind::InvalidData,
format!("Bad schema file stem: {:?}", i)))?
.to_str().ok_or_else(
|| io::Error::new(io::ErrorKind::InvalidData,
format!("Invalid UTF-8 in schema file name: {:?}", i)))?;
bundle.insert(vec![prefix.to_owned()], s);
} else if let Ok(Bundle { modules }) = language.parse(&blob) {
for (ModulePath(k), v) in modules.0 {
@ -126,7 +166,7 @@ pub fn load_schema_or_bundle(bundle: &mut Map<ModulePath, Schema>, i: &PathBuf)
}
} else {
return Err(io::Error::new(io::ErrorKind::InvalidData,
format!("Invalid schema binary blob {:?}", i)));
format!("Invalid schema binary blob {:?}", prefix)));
}
Ok(())
@ -159,18 +199,26 @@ impl CompilerConfig {
}
}
pub fn load_schemas_and_bundles(&mut self, inputs: &Vec<PathBuf>) -> io::Result<()> {
pub fn load_schemas_and_bundles(&mut self, inputs: &Vec<PathBuf>, xrefs: &Vec<PathBuf>) -> io::Result<()> {
for i in inputs {
load_schema_or_bundle(&mut self.bundle, i)?;
load_schema_or_bundle_with_purpose(&mut self.bundle, i, Purpose::Codegen)?;
}
for i in xrefs {
load_schema_or_bundle_with_purpose(&mut self.bundle, i, Purpose::Xref)?;
}
Ok(())
}
pub fn load_xref_bin(&mut self, prefix: &str, bundle_or_schema: &[u8]) -> io::Result<()> {
load_schema_or_bundle_bin_with_purpose(
&mut self.bundle, prefix, bundle_or_schema, Purpose::Xref)
}
fn build_type_cache(&self) -> Map<Ref, types::TDefinition> {
self.bundle.iter().flat_map(|(modpath, s)| {
let modpath = ModulePath(modpath.clone());
s.definitions.0.iter().map(move |(name, def)| {
let ty = types::definition_type(&modpath, name, def);
s.0.definitions.0.iter().map(move |(name, def)| {
let ty = types::definition_type(&modpath, s.1, name, def);
(ty.self_ref.clone(), ty)
})
}).collect()
@ -237,7 +285,11 @@ impl Schema {
pub fn compile(config: &CompilerConfig) -> io::Result<()> {
let mut b = BundleContext::new(config);
for (k, v) in config.bundle.iter() {
for (k, (v, module_purpose)) in config.bundle.iter() {
if *module_purpose != Purpose::Codegen {
continue;
}
let mut output_path = config.output_dir.clone();
output_path.extend(k);
let module_name = output_path.file_stem().unwrap().to_str().unwrap().to_owned();
@ -299,7 +351,10 @@ pub fn compile(config: &CompilerConfig) -> io::Result<()> {
mod_rs.extend(vec!["mod.rs"]);
let mut lines = Vec::new();
for modpath in config.bundle.keys() {
for (modpath, (_, module_purpose)) in config.bundle.iter() {
if *module_purpose != Purpose::Codegen {
continue;
}
lines.push(format!("pub mod {};", names::render_modname(modpath.last().unwrap())));
}
lines.push("".to_owned());
@ -329,6 +384,31 @@ pub fn compile(config: &CompilerConfig) -> io::Result<()> {
]
])));
lines.push("".to_owned());
{
let mut b = Bundle { modules: Modules(Map::new()) };
for (modpath, (schema, purpose)) in config.bundle.iter() {
if *purpose == Purpose::Codegen {
b.modules.0.insert(ModulePath(modpath.clone()), schema.clone());
}
}
let b_value = Language::default().unparse(&b);
let b_bin = preserves::value::PackedWriter::encode_iovalue(&b_value).unwrap();
let mut hex_encoded_bundle = String::new();
let mut count = 0;
for b in b_bin {
if count % 16 == 0 {
hex_encoded_bundle.push_str("\\\n ");
}
count += 1;
hex_encoded_bundle.push_str(&format!("\\x{:02x}", b));
}
lines.push(Formatter::to_string(item(seq![
"pub fn _bundle() -> &'static [u8] ", codeblock![
seq!["b\"", hex_encoded_bundle, "\""]
]
])));
}
lines.push("".to_owned());
let contents = lines.join("\n");
write_if_changed(&mod_rs, contents.as_bytes())?;

2
implementations/rust/preserves-schema/src/compiler/parsers.rs
View File

@ -22,7 +22,7 @@ impl compiler::Plugin for ParserPlugin {
}
pub fn gen_definition_parser(m: &mut ModuleContext, n: &str, d: &Definition) {
let ty = definition_type(&m.module_path, n, d);
let ty = definition_type(&m.module_path, Purpose::Codegen, n, d);
m.define_function(
n,

5
implementations/rust/preserves-schema/src/compiler/readers.rs
View File

@ -78,7 +78,7 @@ impl BoundaryTracker {
}
pub fn gen_definition_reader(m: &mut ModuleContext, n: &str, d: &Definition) {
let ty = definition_type(&m.module_path, n, d);
let ty = definition_type(&m.module_path, Purpose::Codegen, n, d);
m.define_function(
n,
@ -368,10 +368,7 @@ fn simple_pattern_reader(
dest
},
SimplePattern::Embedded { .. } => {
// TODO: Is this right? If so, why doesn't it expect *two* Embedded tags in a row??
body.push(item("r.reader.open_embedded()?;"));
ctxt.define_atom(body, &dest, item("r.demand_next()?.value().to_embedded()?.clone()"));
body.push(item("r.reader.close_embedded()?;"));
dest
},
SimplePattern::Lit { value } => {

16
implementations/rust/preserves-schema/src/compiler/types.rs
View File

@ -11,8 +11,15 @@ use super::context::ModuleContextMode;
use super::context::RefRenderStyle;
use super::names;
#[derive(Debug, PartialEq, Eq, Clone, Copy, PartialOrd, Ord)]
pub enum Purpose {
Codegen,
Xref,
}
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Clone)]
pub struct TDefinition {
pub purpose: Purpose,
pub self_ref: Ref,
pub body: TDefinitionBody,
}
@ -60,7 +67,7 @@ impl compiler::Plugin for TypePlugin {
fn generate_definition(&self, m: &mut ModuleContext, n: &str, d: &Definition) {
if let ModuleContextMode::TargetGeneric = m.mode {
let ty = definition_type(&m.module_path, n, d);
let ty = definition_type(&m.module_path, Purpose::Codegen, n, d);
m.define_type(item(ty.render(m, n)));
if ty.has_embedded(m.bundle) {
m.define_type(item(
@ -87,8 +94,9 @@ impl compiler::Plugin for TypePlugin {
}
}
pub fn definition_type(module: &ModulePath, n: &str, d: &Definition) -> TDefinition {
pub fn definition_type(module: &ModulePath, purpose: Purpose, n: &str, d: &Definition) -> TDefinition {
TDefinition {
purpose,
self_ref: Ref { module: module.clone(), name: n.to_owned() },
body: match d {
Definition::Or { pattern_0, pattern_1, pattern_n } =>
@ -247,8 +255,8 @@ impl TField {
r,
|ctxt, r| ctxt.type_for_name(r),
|s, t| match t {
Some(ty) => ty._language_types(s, ts),
None => {
Some(ty) if ty.purpose == Purpose::Codegen => ty._language_types(s, ts),
Some(_) | None => {
let xmts = &s.context.config.external_modules.get(&r.module.0).unwrap()
.rust_language_types;
if let Some(f) = xmts.definitions.get(&r.name).or(xmts.fallback.as_ref()) {

10
implementations/rust/preserves-schema/src/compiler/unparsers.rs
View File

@ -53,15 +53,15 @@ struct ValueContext {
}
fn normal_none(is_struct: bool) -> ValueContext {
ValueContext { src: None, sink: ValueSink::Normal, is_struct: is_struct }
ValueContext { src: None, sink: ValueSink::Normal, is_struct }
}
fn normal_src(src: String, is_struct: bool) -> ValueContext {
ValueContext { src: Some(src), sink: ValueSink::Normal, is_struct: is_struct }
ValueContext { src: Some(src), sink: ValueSink::Normal, is_struct }
}
pub fn gen_definition_unparser(m: &mut ModuleContext, n: &str, d: &Definition) {
let ty = definition_type(&m.module_path, n, d);
let ty = definition_type(&m.module_path, Purpose::Codegen, n, d);
m.define_function(
n,
@ -234,7 +234,7 @@ fn pattern_unparser(
sink: ValueSink::Fields(Rc::new(Cell::new(Some(FieldsSink {
finish: item(seq![rtmp.clone(), ".finish().wrap()"]),
vec_expr: item(seq![rtmp.clone(), ".fields_vec_mut()"]),
body: body,
body,
})))),
is_struct: vc.is_struct,
})
@ -285,7 +285,7 @@ fn sequenceify<'a>(
finish: item(seq![
"_support::preserves::value::Value::Sequence", parens![rtmp.clone()], ".wrap()"]),
vec_expr: item(rtmp),
body: body,
body,
}
}
}

189
implementations/rust/preserves-schema/src/gen/mod.rs
View File

@ -70,3 +70,192 @@ impl<N: _support::preserves::value::NestedValue> Default for Language<N> {
}
}
}
pub fn _bundle() -> &'static [u8] {
b"\
\xb4\xb3\x06\x62\x75\x6e\x64\x6c\x65\xb7\xb5\xb3\x06\x73\x63\x68\
\x65\x6d\x61\x84\xb4\xb3\x06\x73\x63\x68\x65\x6d\x61\xb7\xb3\x07\
\x76\x65\x72\x73\x69\x6f\x6e\x91\xb3\x0b\x64\x65\x66\x69\x6e\x69\
\x74\x69\x6f\x6e\x73\xb7\xb3\x03\x52\x65\x66\xb4\xb3\x03\x72\x65\
\x63\xb4\xb3\x03\x6c\x69\x74\xb3\x03\x72\x65\x66\x84\xb4\xb3\x05\
\x74\x75\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x06\
\x6d\x6f\x64\x75\x6c\x65\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x0a\
\x4d\x6f\x64\x75\x6c\x65\x50\x61\x74\x68\x84\x84\xb4\xb3\x05\x6e\
\x61\x6d\x65\x64\xb3\x04\x6e\x61\x6d\x65\xb4\xb3\x04\x61\x74\x6f\
\x6d\xb3\x06\x53\x79\x6d\x62\x6f\x6c\x84\x84\x84\x84\x84\xb3\x06\
\x42\x75\x6e\x64\x6c\x65\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\x6c\
\x69\x74\xb3\x06\x62\x75\x6e\x64\x6c\x65\x84\xb4\xb3\x05\x74\x75\
\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x07\x6d\x6f\
\x64\x75\x6c\x65\x73\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x07\x4d\
\x6f\x64\x75\x6c\x65\x73\x84\x84\x84\x84\x84\xb3\x06\x53\x63\x68\
\x65\x6d\x61\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\x6c\x69\x74\xb3\
\x06\x73\x63\x68\x65\x6d\x61\x84\xb4\xb3\x05\x74\x75\x70\x6c\x65\
\xb5\xb4\xb3\x04\x64\x69\x63\x74\xb7\xb3\x07\x76\x65\x72\x73\x69\
\x6f\x6e\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x07\x76\x65\x72\x73\
\x69\x6f\x6e\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x07\x56\x65\x72\
\x73\x69\x6f\x6e\x84\x84\xb3\x0b\x64\x65\x66\x69\x6e\x69\x74\x69\
\x6f\x6e\x73\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x0b\x64\x65\x66\
\x69\x6e\x69\x74\x69\x6f\x6e\x73\xb4\xb3\x03\x72\x65\x66\xb5\x84\
\xb3\x0b\x44\x65\x66\x69\x6e\x69\x74\x69\x6f\x6e\x73\x84\x84\xb3\
\x0c\x65\x6d\x62\x65\x64\x64\x65\x64\x54\x79\x70\x65\xb4\xb3\x05\
\x6e\x61\x6d\x65\x64\xb3\x0c\x65\x6d\x62\x65\x64\x64\x65\x64\x54\
\x79\x70\x65\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x10\x45\x6d\x62\
\x65\x64\x64\x65\x64\x54\x79\x70\x65\x4e\x61\x6d\x65\x84\x84\x84\
\x84\x84\x84\x84\xb3\x07\x42\x69\x6e\x64\x69\x6e\x67\xb4\xb3\x03\
\x72\x65\x63\xb4\xb3\x03\x6c\x69\x74\xb3\x05\x6e\x61\x6d\x65\x64\
\x84\xb4\xb3\x05\x74\x75\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\
\x65\x64\xb3\x04\x6e\x61\x6d\x65\xb4\xb3\x04\x61\x74\x6f\x6d\xb3\
\x06\x53\x79\x6d\x62\x6f\x6c\x84\x84\xb4\xb3\x05\x6e\x61\x6d\x65\
\x64\xb3\x07\x70\x61\x74\x74\x65\x72\x6e\xb4\xb3\x03\x72\x65\x66\
\xb5\x84\xb3\x0d\x53\x69\x6d\x70\x6c\x65\x50\x61\x74\x74\x65\x72\
\x6e\x84\x84\x84\x84\x84\xb3\x07\x4d\x6f\x64\x75\x6c\x65\x73\xb4\
\xb3\x06\x64\x69\x63\x74\x6f\x66\xb4\xb3\x03\x72\x65\x66\xb5\x84\
\xb3\x0a\x4d\x6f\x64\x75\x6c\x65\x50\x61\x74\x68\x84\xb4\xb3\x03\
\x72\x65\x66\xb5\x84\xb3\x06\x53\x63\x68\x65\x6d\x61\x84\x84\xb3\
\x07\x50\x61\x74\x74\x65\x72\x6e\xb4\xb3\x02\x6f\x72\xb5\xb5\xb1\
\x0d\x53\x69\x6d\x70\x6c\x65\x50\x61\x74\x74\x65\x72\x6e\xb4\xb3\
\x03\x72\x65\x66\xb5\x84\xb3\x0d\x53\x69\x6d\x70\x6c\x65\x50\x61\
\x74\x74\x65\x72\x6e\x84\x84\xb5\xb1\x0f\x43\x6f\x6d\x70\x6f\x75\
\x6e\x64\x50\x61\x74\x74\x65\x72\x6e\xb4\xb3\x03\x72\x65\x66\xb5\
\x84\xb3\x0f\x43\x6f\x6d\x70\x6f\x75\x6e\x64\x50\x61\x74\x74\x65\
\x72\x6e\x84\x84\x84\x84\xb3\x07\x56\x65\x72\x73\x69\x6f\x6e\xb4\
\xb3\x03\x6c\x69\x74\x91\x84\xb3\x08\x41\x74\x6f\x6d\x4b\x69\x6e\
\x64\xb4\xb3\x02\x6f\x72\xb5\xb5\xb1\x07\x42\x6f\x6f\x6c\x65\x61\
\x6e\xb4\xb3\x03\x6c\x69\x74\xb3\x07\x42\x6f\x6f\x6c\x65\x61\x6e\
\x84\x84\xb5\xb1\x05\x46\x6c\x6f\x61\x74\xb4\xb3\x03\x6c\x69\x74\
\xb3\x05\x46\x6c\x6f\x61\x74\x84\x84\xb5\xb1\x06\x44\x6f\x75\x62\
\x6c\x65\xb4\xb3\x03\x6c\x69\x74\xb3\x06\x44\x6f\x75\x62\x6c\x65\
\x84\x84\xb5\xb1\x0d\x53\x69\x67\x6e\x65\x64\x49\x6e\x74\x65\x67\
\x65\x72\xb4\xb3\x03\x6c\x69\x74\xb3\x0d\x53\x69\x67\x6e\x65\x64\
\x49\x6e\x74\x65\x67\x65\x72\x84\x84\xb5\xb1\x06\x53\x74\x72\x69\
\x6e\x67\xb4\xb3\x03\x6c\x69\x74\xb3\x06\x53\x74\x72\x69\x6e\x67\
\x84\x84\xb5\xb1\x0a\x42\x79\x74\x65\x53\x74\x72\x69\x6e\x67\xb4\
\xb3\x03\x6c\x69\x74\xb3\x0a\x42\x79\x74\x65\x53\x74\x72\x69\x6e\
\x67\x84\x84\xb5\xb1\x06\x53\x79\x6d\x62\x6f\x6c\xb4\xb3\x03\x6c\
\x69\x74\xb3\x06\x53\x79\x6d\x62\x6f\x6c\x84\x84\x84\x84\xb3\x0a\
\x44\x65\x66\x69\x6e\x69\x74\x69\x6f\x6e\xb4\xb3\x02\x6f\x72\xb5\
\xb5\xb1\x02\x6f\x72\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\x6c\x69\
\x74\xb3\x02\x6f\x72\x84\xb4\xb3\x05\x74\x75\x70\x6c\x65\xb5\xb4\
\xb3\x0b\x74\x75\x70\x6c\x65\x50\x72\x65\x66\x69\x78\xb5\xb4\xb3\
\x05\x6e\x61\x6d\x65\x64\xb3\x08\x70\x61\x74\x74\x65\x72\x6e\x30\
\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x10\x4e\x61\x6d\x65\x64\x41\
\x6c\x74\x65\x72\x6e\x61\x74\x69\x76\x65\x84\x84\xb4\xb3\x05\x6e\
\x61\x6d\x65\x64\xb3\x08\x70\x61\x74\x74\x65\x72\x6e\x31\xb4\xb3\
\x03\x72\x65\x66\xb5\x84\xb3\x10\x4e\x61\x6d\x65\x64\x41\x6c\x74\
\x65\x72\x6e\x61\x74\x69\x76\x65\x84\x84\x84\xb4\xb3\x05\x6e\x61\
\x6d\x65\x64\xb3\x08\x70\x61\x74\x74\x65\x72\x6e\x4e\xb4\xb3\x05\
\x73\x65\x71\x6f\x66\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x10\x4e\
\x61\x6d\x65\x64\x41\x6c\x74\x65\x72\x6e\x61\x74\x69\x76\x65\x84\
\x84\x84\x84\x84\x84\x84\x84\xb5\xb1\x03\x61\x6e\x64\xb4\xb3\x03\
\x72\x65\x63\xb4\xb3\x03\x6c\x69\x74\xb3\x03\x61\x6e\x64\x84\xb4\
\xb3\x05\x74\x75\x70\x6c\x65\xb5\xb4\xb3\x0b\x74\x75\x70\x6c\x65\
\x50\x72\x65\x66\x69\x78\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\
\x08\x70\x61\x74\x74\x65\x72\x6e\x30\xb4\xb3\x03\x72\x65\x66\xb5\
\x84\xb3\x0c\x4e\x61\x6d\x65\x64\x50\x61\x74\x74\x65\x72\x6e\x84\
\x84\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x08\x70\x61\x74\x74\x65\
\x72\x6e\x31\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x0c\x4e\x61\x6d\
\x65\x64\x50\x61\x74\x74\x65\x72\x6e\x84\x84\x84\xb4\xb3\x05\x6e\
\x61\x6d\x65\x64\xb3\x08\x70\x61\x74\x74\x65\x72\x6e\x4e\xb4\xb3\
\x05\x73\x65\x71\x6f\x66\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x0c\
\x4e\x61\x6d\x65\x64\x50\x61\x74\x74\x65\x72\x6e\x84\x84\x84\x84\
\x84\x84\x84\x84\xb5\xb1\x07\x50\x61\x74\x74\x65\x72\x6e\xb4\xb3\
\x03\x72\x65\x66\xb5\x84\xb3\x07\x50\x61\x74\x74\x65\x72\x6e\x84\
\x84\x84\x84\xb3\x0a\x4d\x6f\x64\x75\x6c\x65\x50\x61\x74\x68\xb4\
\xb3\x05\x73\x65\x71\x6f\x66\xb4\xb3\x04\x61\x74\x6f\x6d\xb3\x06\
\x53\x79\x6d\x62\x6f\x6c\x84\x84\xb3\x0b\x44\x65\x66\x69\x6e\x69\
\x74\x69\x6f\x6e\x73\xb4\xb3\x06\x64\x69\x63\x74\x6f\x66\xb4\xb3\
\x04\x61\x74\x6f\x6d\xb3\x06\x53\x79\x6d\x62\x6f\x6c\x84\xb4\xb3\
\x03\x72\x65\x66\xb5\x84\xb3\x0a\x44\x65\x66\x69\x6e\x69\x74\x69\
\x6f\x6e\x84\x84\xb3\x0c\x4e\x61\x6d\x65\x64\x50\x61\x74\x74\x65\
\x72\x6e\xb4\xb3\x02\x6f\x72\xb5\xb5\xb1\x05\x6e\x61\x6d\x65\x64\
\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x07\x42\x69\x6e\x64\x69\x6e\
\x67\x84\x84\xb5\xb1\x09\x61\x6e\x6f\x6e\x79\x6d\x6f\x75\x73\xb4\
\xb3\x03\x72\x65\x66\xb5\x84\xb3\x07\x50\x61\x74\x74\x65\x72\x6e\
\x84\x84\x84\x84\xb3\x0d\x53\x69\x6d\x70\x6c\x65\x50\x61\x74\x74\
\x65\x72\x6e\xb4\xb3\x02\x6f\x72\xb5\xb5\xb1\x03\x61\x6e\x79\xb4\
\xb3\x03\x6c\x69\x74\xb3\x03\x61\x6e\x79\x84\x84\xb5\xb1\x04\x61\
\x74\x6f\x6d\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\x6c\x69\x74\xb3\
\x04\x61\x74\x6f\x6d\x84\xb4\xb3\x05\x74\x75\x70\x6c\x65\xb5\xb4\
\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x08\x61\x74\x6f\x6d\x4b\x69\x6e\
\x64\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x08\x41\x74\x6f\x6d\x4b\
\x69\x6e\x64\x84\x84\x84\x84\x84\x84\xb5\xb1\x08\x65\x6d\x62\x65\
\x64\x64\x65\x64\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\x6c\x69\x74\
\xb3\x08\x65\x6d\x62\x65\x64\x64\x65\x64\x84\xb4\xb3\x05\x74\x75\
\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x09\x69\x6e\
\x74\x65\x72\x66\x61\x63\x65\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\
\x0d\x53\x69\x6d\x70\x6c\x65\x50\x61\x74\x74\x65\x72\x6e\x84\x84\
\x84\x84\x84\x84\xb5\xb1\x03\x6c\x69\x74\xb4\xb3\x03\x72\x65\x63\
\xb4\xb3\x03\x6c\x69\x74\xb3\x03\x6c\x69\x74\x84\xb4\xb3\x05\x74\
\x75\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x05\x76\
\x61\x6c\x75\x65\xb3\x03\x61\x6e\x79\x84\x84\x84\x84\x84\xb5\xb1\
\x05\x73\x65\x71\x6f\x66\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\x6c\
\x69\x74\xb3\x05\x73\x65\x71\x6f\x66\x84\xb4\xb3\x05\x74\x75\x70\
\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x07\x70\x61\x74\
\x74\x65\x72\x6e\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x0d\x53\x69\
\x6d\x70\x6c\x65\x50\x61\x74\x74\x65\x72\x6e\x84\x84\x84\x84\x84\
\x84\xb5\xb1\x05\x73\x65\x74\x6f\x66\xb4\xb3\x03\x72\x65\x63\xb4\
\xb3\x03\x6c\x69\x74\xb3\x05\x73\x65\x74\x6f\x66\x84\xb4\xb3\x05\
\x74\x75\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x07\
\x70\x61\x74\x74\x65\x72\x6e\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\
\x0d\x53\x69\x6d\x70\x6c\x65\x50\x61\x74\x74\x65\x72\x6e\x84\x84\
\x84\x84\x84\x84\xb5\xb1\x06\x64\x69\x63\x74\x6f\x66\xb4\xb3\x03\
\x72\x65\x63\xb4\xb3\x03\x6c\x69\x74\xb3\x06\x64\x69\x63\x74\x6f\
\x66\x84\xb4\xb3\x05\x74\x75\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\
\x6d\x65\x64\xb3\x03\x6b\x65\x79\xb4\xb3\x03\x72\x65\x66\xb5\x84\
\xb3\x0d\x53\x69\x6d\x70\x6c\x65\x50\x61\x74\x74\x65\x72\x6e\x84\
\x84\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x05\x76\x61\x6c\x75\x65\
\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x0d\x53\x69\x6d\x70\x6c\x65\
\x50\x61\x74\x74\x65\x72\x6e\x84\x84\x84\x84\x84\x84\xb5\xb1\x03\
\x52\x65\x66\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x03\x52\x65\x66\
\x84\x84\x84\x84\xb3\x0f\x43\x6f\x6d\x70\x6f\x75\x6e\x64\x50\x61\
\x74\x74\x65\x72\x6e\xb4\xb3\x02\x6f\x72\xb5\xb5\xb1\x03\x72\x65\
\x63\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\x6c\x69\x74\xb3\x03\x72\
\x65\x63\x84\xb4\xb3\x05\x74\x75\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\
\x61\x6d\x65\x64\xb3\x05\x6c\x61\x62\x65\x6c\xb4\xb3\x03\x72\x65\
\x66\xb5\x84\xb3\x0c\x4e\x61\x6d\x65\x64\x50\x61\x74\x74\x65\x72\
\x6e\x84\x84\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x06\x66\x69\x65\
\x6c\x64\x73\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x0c\x4e\x61\x6d\
\x65\x64\x50\x61\x74\x74\x65\x72\x6e\x84\x84\x84\x84\x84\x84\xb5\
\xb1\x05\x74\x75\x70\x6c\x65\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\
\x6c\x69\x74\xb3\x05\x74\x75\x70\x6c\x65\x84\xb4\xb3\x05\x74\x75\
\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x08\x70\x61\
\x74\x74\x65\x72\x6e\x73\xb4\xb3\x05\x73\x65\x71\x6f\x66\xb4\xb3\
\x03\x72\x65\x66\xb5\x84\xb3\x0c\x4e\x61\x6d\x65\x64\x50\x61\x74\
\x74\x65\x72\x6e\x84\x84\x84\x84\x84\x84\x84\xb5\xb1\x0b\x74\x75\
\x70\x6c\x65\x50\x72\x65\x66\x69\x78\xb4\xb3\x03\x72\x65\x63\xb4\
\xb3\x03\x6c\x69\x74\xb3\x0b\x74\x75\x70\x6c\x65\x50\x72\x65\x66\
\x69\x78\x84\xb4\xb3\x05\x74\x75\x70\x6c\x65\xb5\xb4\xb3\x05\x6e\
\x61\x6d\x65\x64\xb3\x05\x66\x69\x78\x65\x64\xb4\xb3\x05\x73\x65\
\x71\x6f\x66\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x0c\x4e\x61\x6d\
\x65\x64\x50\x61\x74\x74\x65\x72\x6e\x84\x84\x84\xb4\xb3\x05\x6e\
\x61\x6d\x65\x64\xb3\x08\x76\x61\x72\x69\x61\x62\x6c\x65\xb4\xb3\
\x03\x72\x65\x66\xb5\x84\xb3\x12\x4e\x61\x6d\x65\x64\x53\x69\x6d\
\x70\x6c\x65\x50\x61\x74\x74\x65\x72\x6e\x84\x84\x84\x84\x84\x84\
\xb5\xb1\x04\x64\x69\x63\x74\xb4\xb3\x03\x72\x65\x63\xb4\xb3\x03\
\x6c\x69\x74\xb3\x04\x64\x69\x63\x74\x84\xb4\xb3\x05\x74\x75\x70\
\x6c\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x07\x65\x6e\x74\
\x72\x69\x65\x73\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x11\x44\x69\
\x63\x74\x69\x6f\x6e\x61\x72\x79\x45\x6e\x74\x72\x69\x65\x73\x84\
\x84\x84\x84\x84\x84\x84\x84\xb3\x10\x45\x6d\x62\x65\x64\x64\x65\
\x64\x54\x79\x70\x65\x4e\x61\x6d\x65\xb4\xb3\x02\x6f\x72\xb5\xb5\
\xb1\x05\x66\x61\x6c\x73\x65\xb4\xb3\x03\x6c\x69\x74\x80\x84\x84\
\xb5\xb1\x03\x52\x65\x66\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x03\
\x52\x65\x66\x84\x84\x84\x84\xb3\x10\x4e\x61\x6d\x65\x64\x41\x6c\
\x74\x65\x72\x6e\x61\x74\x69\x76\x65\xb4\xb3\x05\x74\x75\x70\x6c\
\x65\xb5\xb4\xb3\x05\x6e\x61\x6d\x65\x64\xb3\x0c\x76\x61\x72\x69\
\x61\x6e\x74\x4c\x61\x62\x65\x6c\xb4\xb3\x04\x61\x74\x6f\x6d\xb3\
\x06\x53\x74\x72\x69\x6e\x67\x84\x84\xb4\xb3\x05\x6e\x61\x6d\x65\
\x64\xb3\x07\x70\x61\x74\x74\x65\x72\x6e\xb4\xb3\x03\x72\x65\x66\
\xb5\x84\xb3\x07\x50\x61\x74\x74\x65\x72\x6e\x84\x84\x84\x84\xb3\
\x11\x44\x69\x63\x74\x69\x6f\x6e\x61\x72\x79\x45\x6e\x74\x72\x69\
\x65\x73\xb4\xb3\x06\x64\x69\x63\x74\x6f\x66\xb3\x03\x61\x6e\x79\
\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x12\x4e\x61\x6d\x65\x64\x53\
\x69\x6d\x70\x6c\x65\x50\x61\x74\x74\x65\x72\x6e\x84\x84\xb3\x12\
\x4e\x61\x6d\x65\x64\x53\x69\x6d\x70\x6c\x65\x50\x61\x74\x74\x65\
\x72\x6e\xb4\xb3\x02\x6f\x72\xb5\xb5\xb1\x05\x6e\x61\x6d\x65\x64\
\xb4\xb3\x03\x72\x65\x66\xb5\x84\xb3\x07\x42\x69\x6e\x64\x69\x6e\
\x67\x84\x84\xb5\xb1\x09\x61\x6e\x6f\x6e\x79\x6d\x6f\x75\x73\xb4\
\xb3\x03\x72\x65\x66\xb5\x84\xb3\x0d\x53\x69\x6d\x70\x6c\x65\x50\
\x61\x74\x74\x65\x72\x6e\x84\x84\x84\x84\x84\xb3\x0c\x65\x6d\x62\
\x65\x64\x64\x65\x64\x54\x79\x70\x65\x80\x84\x84\x84\x84"
}

13
implementations/rust/preserves-schema/src/lib.rs
View File

@ -70,9 +70,16 @@ macro_rules! define_language {
mod $fname {
use super::*;
lazy_static::lazy_static! {
pub static ref GLOBAL_LANG: $lang<$default_value> = $lang {
$($field: std::sync::Arc::new($($type)::*::default())),*
};
pub static ref GLOBAL_LANG: std::sync::Arc<$lang<$default_value>> =
std::sync::Arc::new($lang {
$($field: std::sync::Arc::new($($type)::*::default())),*
});
}
}
impl $lang<$default_value> {
pub fn arc() -> &'static std::sync::Arc<$lang<$default_value>> {
&*$fname::GLOBAL_LANG
}
}

BIN
logo-256x256.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 60 KiB

BIN
logo-64x64.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.8 KiB

6
preserves-path.md
View File

@ -6,8 +6,12 @@ title: "Preserves Path"
Tony Garnock-Jones <tonyg@leastfixedpoint.com>
August 2021. Version 0.1.0.
XML documents can move into attributes, into text, or into children.
Preserves Path is roughly analogous to
[XPath](https://www.w3.org/TR/2017/REC-xpath-31-20170321/), but for Preserves values: just as
XPath selects portions of an XML document, a Preserves Path uses *path expressions* to select
portions of a `Value`.
XPaths on XML documents can move into attributes, into text, or into children.
Preserves documents don't have attributes, but they do have children
generally and keyed children in particular. You might want to move
into the child with a particular key (number, for sequences, or

27
preserves-schema.md
View File

@ -4,7 +4,7 @@ title: "Preserves Schema"
---
Tony Garnock-Jones <tonyg@leastfixedpoint.com>
June 2022. Version 0.3.0.
February 2023. Version 0.3.1.
[abnf]: https://tools.ietf.org/html/rfc7405
@ -13,22 +13,7 @@ This document proposes a Schema language for the
## Introduction
A Preserves schema connects Preserves `Value`s to host-language data
structures. Each definition within a schema can be processed by a
compiler to produce
- a simple host-language *type definition*;
- a partial *parsing* function from `Value`s to instances of the
produced type; and
- a total *serialization* function from instances of the type to
`Value`s.
Every parsed `Value` retains enough information to always be able to
be serialized again, and every instance of a host-language data
structure contains, by construction, enough information to be
successfully serialized.
{% include what-is-preserves-schema.md %}
**Portability.** Preserves Schema is broadly portable. Any host-language
type system that can represent [algebraic
@ -565,7 +550,7 @@ present, it must be `1`.
An `EmbeddedTypeName` specifies the type of embedded values within
values parsed by a given schema:
EmbeddedTypeName = Ref / #f.
EmbeddedTypeName = #f / Ref .
Ref = <ref @module ModulePath @name symbol>.
The `Definitions` are a named collection of definitions within a
@ -723,8 +708,8 @@ metaschema.
]>,
EmbeddedTypeName: <or [
["Ref", <ref [] Ref>],
["false", <lit #f>]
["false", <lit #f>],
["Ref", <ref [] Ref>]
]>,
ModulePath: <seqof <atom Symbol>>,
@ -803,7 +788,7 @@ definitions for the metaschema.
export type Version = null;
export type EmbeddedTypeName = ({"_variant": "Ref", "value": Ref} | {"_variant": "false"});
export type EmbeddedTypeName = ({"_variant": "false"} | {"_variant": "Ref", "value": Ref});
export type Definitions = _.KeyedDictionary<symbol, Definition, _embedded>;

11
preserves-text.md
View File

@ -54,7 +54,10 @@ label-`Value` followed by its field-`Value`s.
curly-brace-enclosed colon-separated pairs of values. `Set`s are
written as values enclosed by the tokens `#{` and
`}`.[^printing-collections] It is an error for a set to contain
duplicate elements or for a dictionary to contain duplicate keys.
duplicate elements or for a dictionary to contain duplicate keys. When
printing sets and dictionaries, implementations *SHOULD* order
elements resp. keys with respect to the [total order over
`Value`s](preserves.html#total-order).[^rationale-print-ordering]
Sequence = "[" *Value ws "]"
Dictionary = "{" *(Value ws ":" Value) ws "}"
@ -68,6 +71,12 @@ duplicate elements or for a dictionary to contain duplicate keys.
commas separating, and commas terminating elements or key/value
pairs within a collection.
[^rationale-print-ordering]: **Rationale.** Consistently printing
the elements of unordered collections in some arbitrary but stable
order helps, for example, keep diffs small and somewhat meaningful
when Preserves values are pretty-printed to text documents under
source control.
`Boolean`s are the simple literal strings `#t` and `#f` for true and
false, respectively.

95
preserves.css
View File

@ -6,14 +6,97 @@ body {
font-family: var(--serif-font);
box-sizing: border-box;
line-height: 1.414;
margin: 0;
padding: 0;
}
body > nav {
display: flex;
background: #4a8fa3; /* #385c87; */
color: white;
flex-flow: row nowrap;
box-shadow: 0 0 0.2rem #0000001a, 0 0.2rem 0.4rem #00000033;
padding: 0;
position: sticky;
right: 0;
top: 0;
z-index: 4;
}
html {
scroll-padding-top: 5rem;
}
body > nav > div {
display: flex;
flex-flow: row nowrap;
font-size: 1.2rem;
line-height: 2rem;
margin: 0;
padding: 1rem;
}
body > nav > div * {
margin: 0;
padding: 0;
}
nav div {
flex-grow: 1;
}
nav div.middle {
justify-content: flex-end;
flex-grow: 0;
flex-basis: 40em;
}
@media screen and (max-width: 420px) {
nav div.middle {
display: none;
}
}
nav div.right {
justify-content: flex-end;
}
body > nav > * h1 {
font-size: 1.4rem;
font-weight: normal;
color: white;
}
body > nav > * a {
color: inherit;
text-decoration: none;
}
body > nav span.icon > img {
display: inline-block;
min-height: 48px;
max-height: 48px;
margin: -1.2rem 0;
line-height: 0;
}
body > nav ul {
display: flex;
margin: 0 -0.5rem;
}
body > nav ul > li {
display: block;
padding: 0 0.5rem;
}
@media screen and (max-width: 768px) { .md, .lg, .xl { display: none; } }
@media screen and (max-width: 992px) { .lg, .xl { display: none; } }
@media screen and (max-width: 1200px) { .xl { display: none; } }
@media screen {
body { padding-top: 2rem; max-width: 40em; margin: auto; font-size: 120%; }
hr { display: none; }
main {
padding-top: 2rem;
max-width: 40em;
margin: auto;
font-size: 120%;
}
hr {
display: none;
}
}
@media print {
@page { size: letter; margin: 4rem 0rem 4.333rem 0rem; }
body { margin-left: 4.5rem; margin-right: 4.5rem; font-size: 10.5pt; }
body > nav { display: none; }
main { margin-left: 4.5rem; margin-right: 4.5rem; font-size: 10.5pt; }
h1, h2 { page-break-before: always; margin-top: 0; }
hr+* { page-break-before: always; margin-top: 0; }
hr { display: none; }
@ -29,8 +112,8 @@ p, ul, table {
margin: 1em 0;
}
body {
counter-set: section 0 subsection 0 appendix 0;
main {
counter-set: section 0 subsection 0 appendix 0;
}
h2:before, h3:before {
font-size: 75%;
@ -47,7 +130,7 @@ h2:before, h3:before {
}
@media screen and (max-width: 53.33em) {
body {
main {
margin-left: 2.33em;
margin-right: 0.5em;
}

31
preserves.md
View File

@ -6,19 +6,7 @@ title: "Preserves: an Expressive Data Language"
Tony Garnock-Jones <tonyg@leastfixedpoint.com>
{{ site.version_date }}. Version {{ site.version }}.
*Preserves* is a data model, with associated serialization formats.
It supports *records* with user-defined *labels*, embedded *references*,
and the usual suite of atomic and compound data types, including
*binary* data as a distinct type from text strings. Its *annotations*
allow separation of data from metadata such as
[comments](conventions.html#comments), trace information, and provenance
information.
Preserves departs from many other data languages in defining how to
*compare* two values. Comparison is based on the data model, not on
syntax or on data structures of any particular implementation
language.
{% include what-is-preserves.md %}
This document defines the core semantics and data model of Preserves and
presents a handful of examples. Two other core documents define
@ -38,22 +26,7 @@ element of that set.
data. Every `Value` is finite and non-cyclic. Embedded values, called
`Embedded`s, are a third, special-case category.
Value = Atom
| Compound
| Embedded
Atom = Boolean
| Float
| Double
| SignedInteger
| String
| ByteString
| Symbol
Compound = Record
| Sequence
| Set
| Dictionary
{% include value-grammar.md %}
**Total order.**<a name="total-order"></a> As we go, we will
incrementally specify a total order over `Value`s. Two values of the

1
python/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
sitemap.xml.gz

437
python/0.18.0/404.html Normal file
View File

@ -0,0 +1,437 @@
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<link rel="icon" href="/python/latest/assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.4.2, mkdocs-material-9.1.3">
<title>Python Preserves</title>
<link rel="stylesheet" href="/python/latest/assets/stylesheets/main.c4a75a56.min.css">
<link rel="stylesheet" href="/python/latest/assets/stylesheets/palette.a0c5b2b5.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<link rel="stylesheet" href="/python/latest/assets/_mkdocstrings.css">
<script>__md_scope=new URL("/python/latest/",location),__md_hash=e=>[...e].reduce((e,_)=>(e<<5)-e+_.charCodeAt(0),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="" data-md-color-accent="">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
</div>
<div data-md-component="announce">
</div>
<div data-md-color-scheme="default" data-md-component="outdated" hidden>
</div>
<header class="md-header md-header--shadow" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="/python/latest/." title="Python Preserves" class="md-header__button md-logo" aria-label="Python Preserves" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
Python Preserves
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
</span>
</div>
</div>
</div>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41Z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://gitlab.com/preserves/preserves" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.3.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
</div>
<div class="md-source__repository">
GitLab
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="/python/latest/." title="Python Preserves" class="md-nav__button md-logo" aria-label="Python Preserves" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
</a>
Python Preserves
</label>
<div class="md-nav__source">
<a href="https://gitlab.com/preserves/preserves" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.3.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
</div>
<div class="md-source__repository">
GitLab
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/python/latest/." class="md-nav__link">
Overview
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/api/" class="md-nav__link">
The top-level preserves package
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/binary/" class="md-nav__link">
Machine-oriented binary syntax
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/compare/" class="md-nav__link">
Comparing Values
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/error/" class="md-nav__link">
Codec errors
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/fold/" class="md-nav__link">
Traversing values
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/merge/" class="md-nav__link">
Merging values
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/path/" class="md-nav__link">
Preserves Path
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/schema/" class="md-nav__link">
Preserves Schema
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/text/" class="md-nav__link">
Human-readable text syntax
</a>
</li>
<li class="md-nav__item">
<a href="/python/latest/values/" class="md-nav__link">
Representations of Values
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1>404 - Not found</h1>
</article>
</div>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"base": "/python/latest/", "features": [], "search": "/python/latest/assets/javascripts/workers/search.208ed371.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": {"provider": "mike"}}</script>
<script src="/python/latest/assets/javascripts/bundle.efa0ade1.min.js"></script>
</body>
</html>

698
python/0.18.0/api/index.html Normal file
View File

@ -0,0 +1,698 @@
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<link rel="canonical" href="https://preserves.dev/python/latest/api/">
<link rel="prev" href="..">
<link rel="next" href="../binary/">
<link rel="icon" href="../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.4.2, mkdocs-material-9.1.3">
<title>The top-level preserves package - Python Preserves</title>
<link rel="stylesheet" href="../assets/stylesheets/main.c4a75a56.min.css">
<link rel="stylesheet" href="../assets/stylesheets/palette.a0c5b2b5.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<link rel="stylesheet" href="../assets/_mkdocstrings.css">
<script>__md_scope=new URL("..",location),__md_hash=e=>[...e].reduce((e,_)=>(e<<5)-e+_.charCodeAt(0),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="" data-md-color-accent="">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#the-top-level-preserves-package" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
</div>
<div data-md-color-scheme="default" data-md-component="outdated" hidden>
</div>
<header class="md-header md-header--shadow" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href=".." title="Python Preserves" class="md-header__button md-logo" aria-label="Python Preserves" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
Python Preserves
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
The top-level preserves package
</span>
</div>
</div>
</div>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41Z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://gitlab.com/preserves/preserves" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.3.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
</div>
<div class="md-source__repository">
GitLab
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href=".." title="Python Preserves" class="md-nav__button md-logo" aria-label="Python Preserves" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54Z"/></svg>
</a>
Python Preserves
</label>
<div class="md-nav__source">
<a href="https://gitlab.com/preserves/preserves" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.3.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2023 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81z"/></svg>
</div>
<div class="md-source__repository">
GitLab
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href=".." class="md-nav__link">
Overview
</a>
</li>
<li class="md-nav__item md-nav__item--active">
<input class="md-nav__toggle md-toggle" type="checkbox" id="__toc">
<label class="md-nav__link md-nav__link--active" for="__toc">
The top-level preserves package
<span class="md-nav__icon md-icon"></span>
</label>
<a href="./" class="md-nav__link md-nav__link--active">
The top-level preserves package
</a>
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#preserves" class="md-nav__link">
preserves
</a>
</li>
<li class="md-nav__item">
<a href="#preserves.dumps" class="md-nav__link">
dumps
</a>
</li>
<li class="md-nav__item">
<a href="#preserves.loads" class="md-nav__link">
loads
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../binary/" class="md-nav__link">
Machine-oriented binary syntax
</a>
</li>
<li class="md-nav__item">
<a href="../compare/" class="md-nav__link">
Comparing Values
</a>
</li>
<li class="md-nav__item">
<a href="../error/" class="md-nav__link">
Codec errors
</a>
</li>
<li class="md-nav__item">
<a href="../fold/" class="md-nav__link">
Traversing values
</a>
</li>
<li class="md-nav__item">
<a href="../merge/" class="md-nav__link">
Merging values
</a>
</li>
<li class="md-nav__item">
<a href="../path/" class="md-nav__link">
Preserves Path
</a>
</li>
<li class="md-nav__item">
<a href="../schema/" class="md-nav__link">
Preserves Schema
</a>
</li>
<li class="md-nav__item">
<a href="../text/" class="md-nav__link">
Human-readable text syntax
</a>
</li>
<li class="md-nav__item">
<a href="../values/" class="md-nav__link">
Representations of Values
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#preserves" class="md-nav__link">
preserves
</a>
</li>
<li class="md-nav__item">
<a href="#preserves.dumps" class="md-nav__link">
dumps
</a>
</li>
<li class="md-nav__item">
<a href="#preserves.loads" class="md-nav__link">
loads
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1 id="the-top-level-preserves-package">The top-level preserves package</h1>
<div class="doc doc-object doc-module">
<a id="preserves"></a>
<div class="doc doc-contents first">
<div class="highlight"><pre><span></span><code>import preserves
</code></pre></div>
<p>The main package re-exports a subset of the exports of its constituent modules:</p>
<ul>
<li>
<p>From <a class="autorefs autorefs-internal" href="../values/#preserves.values">preserves.values</a>:</p>
<ul>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.Annotated">Annotated</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.Embedded">Embedded</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.Float">Float</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.ImmutableDict">ImmutableDict</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.Record">Record</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.Symbol">Symbol</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.annotate">annotate</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.is_annotated">is_annotated</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.preserve">preserve</a></li>
<li><a class="autorefs autorefs-internal" href="../values/#preserves.values.strip_annotations">strip_annotations</a></li>
</ul>
</li>
<li>
<p>From <a class="autorefs autorefs-internal" href="../error/#preserves.error">preserves.error</a>:</p>
<ul>
<li><a class="autorefs autorefs-internal" href="../error/#preserves.error.DecodeError">DecodeError</a></li>
<li><a class="autorefs autorefs-internal" href="../error/#preserves.error.EncodeError">EncodeError</a></li>
<li><a class="autorefs autorefs-internal" href="../error/#preserves.error.ShortPacket">ShortPacket</a></li>
</ul>
</li>
<li>
<p>From <a class="autorefs autorefs-internal" href="../binary/#preserves.binary">preserves.binary</a>:</p>
<ul>
<li><a class="autorefs autorefs-internal" href="../binary/#preserves.binary.Decoder">Decoder</a></li>
<li><a class="autorefs autorefs-internal" href="../binary/#preserves.binary.Encoder">Encoder</a></li>
<li><a class="autorefs autorefs-internal" href="../binary/#preserves.binary.canonicalize">canonicalize</a></li>
<li><a class="autorefs autorefs-internal" href="../binary/#preserves.binary.decode">decode</a></li>
<li><a class="autorefs autorefs-internal" href="../binary/#preserves.binary.decode_with_annotations">decode_with_annotations</a></li>
<li><a class="autorefs autorefs-internal" href="../binary/#preserves.binary.encode">encode</a></li>
</ul>
</li>
<li>
<p>From <a class="autorefs autorefs-internal" href="../text/#preserves.text">preserves.text</a>:</p>
<ul>
<li><a class="autorefs autorefs-internal" href="../text/#preserves.text.Formatter">Formatter</a></li>
<li><a class="autorefs autorefs-internal" href="../text/#preserves.text.Parser">Parser</a></li>
<li><a class="autorefs autorefs-internal" href="../text/#preserves.text.parse">parse</a></li>
<li><a class="autorefs autorefs-internal" href="../text/#preserves.text.parse_with_annotations">parse_with_annotations</a></li>
<li><a class="autorefs autorefs-internal" href="../text/#preserves.text.stringify">stringify</a></li>
</ul>
</li>
<li>
<p>From <a class="autorefs autorefs-internal" href="../compare/#preserves.compare">preserves.compare</a>:</p>
<ul>
<li><a class="autorefs autorefs-internal" href="../compare/#preserves.compare.cmp">cmp</a></li>
</ul>
</li>
<li>
<p>From <a class="autorefs autorefs-internal" href="../merge/#preserves.merge">preserves.merge</a>:</p>
<ul>
<li><a class="autorefs autorefs-internal" href="../merge/#preserves.merge.merge">merge</a></li>
</ul>
</li>
</ul>
<p>It also exports the <a class="autorefs autorefs-internal" href="../compare/#preserves.compare">compare</a> and <a class="autorefs autorefs-internal" href="../fold/#preserves.fold">fold</a> modules themselves,
permitting patterns like</p>
<div class="highlight"><pre><span></span><code><span class="o">&gt;&gt;&gt;</span> <span class="kn">from</span> <span class="nn">preserves</span> <span class="kn">import</span> <span class="o">*</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">compare</span><span class="o">.</span><span class="n">cmp</span><span class="p">(</span><span class="mi">123</span><span class="p">,</span> <span class="mi">234</span><span class="p">)</span>
<span class="o">-</span><span class="mi">1</span>
</code></pre></div>
<p>Finally, it provides a few utility aliases for common tasks:</p>
<div class="doc doc-children">
<div class="doc doc-object doc-attribute">
<h2 id="preserves.dumps" class="doc doc-heading">
<code class="highlight language-python"><span class="n">dumps</span> <span class="o">=</span> <span class="n">stringify</span></code>
<span class="doc doc-labels">
<small class="doc doc-label doc-label-module-attribute"><code>module-attribute</code></small>
</span>
</h2>
<div class="doc doc-contents ">
<p>This alias for <code>stringify</code> provides a familiar pythonesque name for converting a Preserves <code>Value</code> to a string.</p>
</div>
</div>
<div class="doc doc-object doc-attribute">
<h2 id="preserves.loads" class="doc doc-heading">
<code class="highlight language-python"><span class="n">loads</span> <span class="o">=</span> <span class="n">parse</span></code>
<span class="doc doc-labels">
<small class="doc doc-label doc-label-module-attribute"><code>module-attribute</code></small>
</span>
</h2>
<div class="doc doc-contents ">
<p>This alias for <code>parse</code> provides a familiar pythonesque name for converting a string to a Preserves <code>Value</code>.</p>
</div>
</div>
</div>
</div>
</div>
<hr>
<div class="md-source-file">
<small>
Last update:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">March 16, 2023</span>
<br>
Created:
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">March 16, 2023</span>
</small>
</div>
</article>
</div>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"base": "..", "features": [], "search": "../assets/javascripts/workers/search.208ed371.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": {"provider": "mike"}}</script>
<script src="../assets/javascripts/bundle.efa0ade1.min.js"></script>
</body>
</html>

36
python/0.18.0/assets/_mkdocstrings.css Normal file
View File

@ -0,0 +1,36 @@
/* Don't capitalize names. */
h5.doc-heading {
text-transform: none !important;
}
/* Avoid breaking parameters name, etc. in table cells. */
.doc-contents td code {
word-break: normal !important;
}
/* For pieces of Markdown rendered in table cells. */
.doc-contents td p {
margin-top: 0 !important;
margin-bottom: 0 !important;
}
/* Max width for docstring sections tables. */
.doc .md-typeset__table,
.doc .md-typeset__table table {
display: table !important;
width: 100%;
}
.doc .md-typeset__table tr {
display: table-row;
}
/* Avoid line breaks in rendered fields. */
.field-body p {
display: inline;
}
/* Defaults in Spacy table style. */
.doc-param-default {
float: right;
}

BIN
python/0.18.0/assets/images/favicon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.8 KiB

29
python/0.18.0/assets/javascripts/bundle.efa0ade1.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

8
python/0.18.0/assets/javascripts/bundle.efa0ade1.min.js.map Normal file
View File

File diff suppressed because one or more lines are too long

1
python/0.18.0/assets/javascripts/lunr/min/lunr.ar.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

18
python/0.18.0/assets/javascripts/lunr/min/lunr.da.min.js vendored Normal file
View File

@ -0,0 +1,18 @@
/*!
* Lunr languages, `Danish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.da=function(){this.pipeline.reset(),this.pipeline.add(e.da.trimmer,e.da.stopWordFilter,e.da.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.da.stemmer))},e.da.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.da.trimmer=e.trimmerSupport.generateTrimmer(e.da.wordCharacters),e.Pipeline.registerFunction(e.da.trimmer,"trimmer-da"),e.da.stemmer=function(){var r=e.stemmerSupport.Among,i=e.stemmerSupport.SnowballProgram,n=new function(){function e(){var e,r=f.cursor+3;if(d=f.limit,0<=r&&r<=f.limit){for(a=r;;){if(e=f.cursor,f.in_grouping(w,97,248)){f.cursor=e;break}if(f.cursor=e,e>=f.limit)return;f.cursor++}for(;!f.out_grouping(w,97,248);){if(f.cursor>=f.limit)return;f.cursor++}d=f.cursor,d<a&&(d=a)}}function n(){var e,r;if(f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(c,32),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del();break;case 2:f.in_grouping_b(p,97,229)&&f.slice_del()}}function t(){var e,r=f.limit-f.cursor;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.find_among_b(l,4)?(f.bra=f.cursor,f.limit_backward=e,f.cursor=f.limit-r,f.cursor>f.limit_backward&&(f.cursor--,f.bra=f.cursor,f.slice_del())):f.limit_backward=e)}function s(){var e,r,i,n=f.limit-f.cursor;if(f.ket=f.cursor,f.eq_s_b(2,"st")&&(f.bra=f.cursor,f.eq_s_b(2,"ig")&&f.slice_del()),f.cursor=f.limit-n,f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(m,5),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del(),i=f.limit-f.cursor,t(),f.cursor=f.limit-i;break;case 2:f.slice_from("løs")}}function o(){var e;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.out_grouping_b(w,97,248)?(f.bra=f.cursor,u=f.slice_to(u),f.limit_backward=e,f.eq_v_b(u)&&f.slice_del()):f.limit_backward=e)}var a,d,u,c=[new r("hed",-1,1),new r("ethed",0,1),new r("ered",-1,1),new r("e",-1,1),new r("erede",3,1),new r("ende",3,1),new r("erende",5,1),new r("ene",3,1),new r("erne",3,1),new r("ere",3,1),new r("en",-1,1),new r("heden",10,1),new r("eren",10,1),new r("er",-1,1),new r("heder",13,1),new r("erer",13,1),new r("s",-1,2),new r("heds",16,1),new r("es",16,1),new r("endes",18,1),new r("erendes",19,1),new r("enes",18,1),new r("ernes",18,1),new r("eres",18,1),new r("ens",16,1),new r("hedens",24,1),new r("erens",24,1),new r("ers",16,1),new r("ets",16,1),new r("erets",28,1),new r("et",-1,1),new r("eret",30,1)],l=[new r("gd",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("elig",1,1),new r("els",-1,1),new r("løst",-1,2)],w=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],p=[239,254,42,3,0,0,0,0,0,0,0,0,0,0,0,0,16],f=new i;this.setCurrent=function(e){f.setCurrent(e)},this.getCurrent=function(){return f.getCurrent()},this.stem=function(){var r=f.cursor;return e(),f.limit_backward=r,f.cursor=f.limit,n(),f.cursor=f.limit,t(),f.cursor=f.limit,s(),f.cursor=f.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return n.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}}(),e.Pipeline.registerFunction(e.da.stemmer,"stemmer-da"),e.da.stopWordFilter=e.generateStopWordFilter("ad af alle alt anden at blev blive bliver da de dem den denne der deres det dette dig din disse dog du efter eller en end er et for fra ham han hans har havde have hende hendes her hos hun hvad hvis hvor i ikke ind jeg jer jo kunne man mange med meget men mig min mine mit mod ned noget nogle nu når og også om op os over på selv sig sin sine sit skal skulle som sådan thi til ud under var vi vil ville vor være været".split(" ")),e.Pipeline.registerFunction(e.da.stopWordFilter,"stopWordFilter-da")}});

18
python/0.18.0/assets/javascripts/lunr/min/lunr.de.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

18
python/0.18.0/assets/javascripts/lunr/min/lunr.du.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

18
python/0.18.0/assets/javascripts/lunr/min/lunr.es.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

18
python/0.18.0/assets/javascripts/lunr/min/lunr.fi.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

18
python/0.18.0/assets/javascripts/lunr/min/lunr.fr.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

1
python/0.18.0/assets/javascripts/lunr/min/lunr.hi.min.js vendored Normal file
View File

@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hi=function(){this.pipeline.reset(),this.pipeline.add(e.hi.trimmer,e.hi.stopWordFilter,e.hi.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.hi.stemmer))},e.hi.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿa-zA-Zａ-ｚＡ-Ｚ0-9０-９",e.hi.trimmer=e.trimmerSupport.generateTrimmer(e.hi.wordCharacters),e.Pipeline.registerFunction(e.hi.trimmer,"trimmer-hi"),e.hi.stopWordFilter=e.generateStopWordFilter("अत अपना अपनी अपने अभी अंदर आदि आप इत्यादि इन इनका इन्हीं इन्हें इन्हों इस इसका इसकी इसके इसमें इसी इसे उन उनका उनकी उनके उनको उन्हीं उन्हें उन्हों उस उसके उसी उसे एक एवं एस ऐसे और कई कर करता करते करना करने करें कहते कहा का काफ़ी कि कितना किन्हें किन्हों किया किर किस किसी किसे की कुछ कुल के को कोई कौन कौनसा गया घर जब जहाँ जा जितना जिन जिन्हें जिन्हों जिस जिसे जीधर जैसा जैसे जो तक तब तरह तिन तिन्हें तिन्हों तिस तिसे तो था थी थे दबारा दिया दुसरा दूसरे दो द्वारा न नके नहीं ना निहायत नीचे ने पर पहले पूरा पे फिर बनी बही बहुत बाद बाला बिलकुल भी भीतर मगर मानो मे में यदि यह यहाँ यही या यिह ये रखें रहा रहे ऱ्वासा लिए लिये लेकिन व वग़ैरह वर्ग वह वहाँ वहीं वाले वुह वे वो सकता सकते सबसे सभी साथ साबुत साभ सारा से सो संग ही हुआ हुई हुए है हैं हो होता होती होते होना होने".split(" ")),e.hi.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.hi.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var t=i.toString().toLowerCase().replace(/^\s+/,"");return r.cut(t).split("|")},e.Pipeline.registerFunction(e.hi.stemmer,"stemmer-hi"),e.Pipeline.registerFunction(e.hi.stopWordFilter,"stopWordFilter-hi")}});

18
python/0.18.0/assets/javascripts/lunr/min/lunr.hu.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

18
python/0.18.0/assets/javascripts/lunr/min/lunr.it.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

1
python/0.18.0/assets/javascripts/lunr/min/lunr.ja.min.js vendored Normal file
View File

@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.ja=function(){this.pipeline.reset(),this.pipeline.add(e.ja.trimmer,e.ja.stopWordFilter,e.ja.stemmer),r?this.tokenizer=e.ja.tokenizer:(e.tokenizer&&(e.tokenizer=e.ja.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.ja.tokenizer))};var t=new e.TinySegmenter;e.ja.tokenizer=function(i){var n,o,s,p,a,u,m,l,c,f;if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t.toLowerCase()):t.toLowerCase()});for(o=i.toString().toLowerCase().replace(/^\s+/,""),n=o.length-1;n>=0;n--)if(/\S/.test(o.charAt(n))){o=o.substring(0,n+1);break}for(a=[],s=o.length,c=0,l=0;c<=s;c++)if(u=o.charAt(c),m=c-l,u.match(/\s/)||c==s){if(m>0)for(p=t.segment(o.slice(l,c)).filter(function(e){return!!e}),f=l,n=0;n<p.length;n++)r?a.push(new e.Token(p[n],{position:[f,p[n].length],index:a.length})):a.push(p[n]),f+=p[n].length;l=c+1}return a},e.ja.stemmer=function(){return function(e){return e}}(),e.Pipeline.registerFunction(e.ja.stemmer,"stemmer-ja"),e.ja.wordCharacters="一二三四五六七八九十百千万億兆一-龠々〆ヵヶぁ-んァ-ヴーｱ-ﾝﾞa-zA-Zａ-ｚＡ-Ｚ0-9０-９",e.ja.trimmer=e.trimmerSupport.generateTrimmer(e.ja.wordCharacters),e.Pipeline.registerFunction(e.ja.trimmer,"trimmer-ja"),e.ja.stopWordFilter=e.generateStopWordFilter("これ それ あれ この その あの ここ そこ あそこ こちら どこ だれ なに なん 何 私 貴方 貴方方 我々 私達 あの人 あのかた 彼女 彼 です あります おります います は が の に を で え から まで より も どの と し それで しかし".split(" ")),e.Pipeline.registerFunction(e.ja.stopWordFilter,"stopWordFilter-ja"),e.jp=e.ja,e.Pipeline.registerFunction(e.jp.stemmer,"stemmer-jp"),e.Pipeline.registerFunction(e.jp.trimmer,"trimmer-jp"),e.Pipeline.registerFunction(e.jp.stopWordFilter,"stopWordFilter-jp")}});

1
python/0.18.0/assets/javascripts/lunr/min/lunr.jp.min.js vendored Normal file
View File

@ -0,0 +1 @@
module.exports=require("./lunr.ja");

1
python/0.18.0/assets/javascripts/lunr/min/lunr.ko.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

1
python/0.18.0/assets/javascripts/lunr/min/lunr.multi.min.js vendored Normal file
View File

@ -0,0 +1 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){e.multiLanguage=function(){for(var t=Array.prototype.slice.call(arguments),i=t.join("-"),r="",n=[],s=[],p=0;p<t.length;++p)"en"==t[p]?(r+="\\w",n.unshift(e.stopWordFilter),n.push(e.stemmer),s.push(e.stemmer)):(r+=e[t[p]].wordCharacters,e[t[p]].stopWordFilter&&n.unshift(e[t[p]].stopWordFilter),e[t[p]].stemmer&&(n.push(e[t[p]].stemmer),s.push(e[t[p]].stemmer)));var o=e.trimmerSupport.generateTrimmer(r);return e.Pipeline.registerFunction(o,"lunr-multi-trimmer-"+i),n.unshift(o),function(){this.pipeline.reset(),this.pipeline.add.apply(this.pipeline,n),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add.apply(this.searchPipeline,s))}}}});

Some files were not shown because too many files have changed in this diff Show More