syndicate-lang
/
preserves
mirror of https://gitlab.com/preserves/preserves.git
1
0
Fork 1
Code Releases Activity

Switch to "# "-prefixed text comment annotation syntax

This commit is contained in:
Tony Garnock-Jones 2023-10-15 15:11:27 +02:00
parent 4e471ed896
commit ccf277cddb
20 changed files with 375 additions and 363 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
TUTORIAL.md
View File

@ -53,7 +53,7 @@ restricted to strings; more on this later, but for now, we will stick
to the special comment annotation syntax.)
```
;I'm an annotation... basically a comment. Ignore me!
# I'm an annotation... basically a comment. Ignore me!
"I'm data! Don't ignore me!"
```
@ -61,23 +61,23 @@ Preserves supports some data types you're probably already familiar
with from JSON, and which look fairly similar in the textual format:
```
;booleans
# booleans
#t
#f
;various kinds of numbers:
# various kinds of numbers:
42
123556789012345678901234567890
-10
13.5
;strings
# strings
"I'm feeling stringy!"
;sequences (lists)
# sequences (lists)
["cat", "dog", "mouse", "goldfish"]
;dictionaries (hashmaps)
# dictionaries (hashmaps)
{"cat": "meow",
"dog": "woof",
"goldfish": "glub glub",
@ -95,11 +95,11 @@ bit different.
A few more interesting differences:
```
;Preserves treats commas as whitespace, so these are the same
# Preserves treats commas as whitespace, so these are the same
["cat", "dog", "mouse", "goldfish"]
["cat" "dog" "mouse" "goldfish"]
;We can use anything as keys in dictionaries, not just strings
# We can use anything as keys in dictionaries, not just strings
{1: "the loneliest number",
["why", "was", 6, "afraid", "of", 7]: "because 7 8 9",
{"dictionaries": "as keys???"}: "well, why not?"}
@ -108,16 +108,16 @@ A few more interesting differences:
Preserves technically provides a few types of numbers:
```
;Signed Integers
# Signed Integers
42
-42
5907212309572059846509324862304968273468909473609826340
-5907212309572059846509324862304968273468909473609826340
;Floats (Single-precision IEEE floats) (notice the trailing f)
# Floats (Single-precision IEEE floats) (notice the trailing f)
3.1415927f
;Doubles (Double-precision IEEE floats)
# Doubles (Double-precision IEEE floats)
3.141592653589793
```
@ -130,31 +130,31 @@ to the program, but not textual importance (other than to guide the
programmer… not unlike variable names).
```
;A symbol (NOT a string!)
# A symbol (NOT a string!)
JustASymbol
;You can do mixedCase or CamelCase too of course, pick your poison
;(but be consistent, for the sake of your collaborators!)
# You can do mixedCase or CamelCase too of course, pick your poison
# (but be consistent, for the sake of your collaborators!)
iAmASymbol
i-am-a-symbol
;A list of symbols
# A list of symbols
[GET, PUT, POST, DELETE]
;A symbol with spaces in it
# A symbol with spaces in it
|this is just one symbol believe it or not|
```
We can also add binary data, aka ByteStrings:
```
;Some binary data, base64 encoded
# Some binary data, base64 encoded
#[cGljdHVyZSBvZiBhIGNhdA==]
;Some other binary data, hexadecimal encoded
# Some other binary data, hexadecimal encoded
#x"616263"
;Same binary data as above, base64 encoded
# Same binary data as above, base64 encoded
#[YWJj]
```
@ -186,14 +186,14 @@ useful for many contexts, but especially for cryptographic signatures
and hashing.
```
;This hand-typed Preserves document...
# This hand-typed Preserves document...
{monkey: {"noise": "ooh-ooh",
"eats": #{"bananas", "berries"}}
cat: {"noise": "meow",
"eats": #{"kibble", "cat treats", "tinned meat"}}}
;Will always, always be written out in this order (except in
;binary, of course) when canonicalized:
# Will always, always be written out in this order (except in
# binary, of course) when canonicalized:
{cat: {"eats": #{"cat treats", "kibble", "tinned meat"},
"noise": "meow"}
monkey: {"eats": #{"bananas", "berries"},
@ -237,12 +237,12 @@ This causes a problem.
Now we might have two kinds of entries:
```
;Exact date known
# Exact date known
{"name": "Gregor Samsa",
"description": "humanoid trapped in an insect body",
"born": "1915-10-04"}
;Not sure about exact date...
# Not sure about exact date...
{"name": "Gregor Samsa",
"description": "humanoid trapped in an insect body",
"born": "Sometime in October 1915? Or was that when he became an insect?"}
@ -255,12 +255,12 @@ edge cases.
No, it's better to be able to have a separate type:
```
;Exact date known
# Exact date known
{"name": "Gregor Samsa",
"description": "humanoid trapped in an insect body",
"born": <Date 1915 10 04>}
;Not sure about exact date...
# Not sure about exact date...
{"name": "Gregor Samsa",
"description": "humanoid trapped in an insect body",
"born": <Unknown "Sometime in October 1915? Or was that when he became an insect?">}
@ -321,17 +321,17 @@ in some circumstances.
We have previously shown them used as comments:
```
;I'm a comment!
# I'm a comment!
"I am not a comment, I am data!"
```
Annotations annotate the values they precede.
It is possible to have multiple annotations on a value.
The `;`-based comment syntax is syntactic sugar for the general
The hash-space (or hash-tab) comment syntax is syntactic sugar for the general
`@`-prefixed string annotation syntax.
```
;I am annotating this number
# I am annotating this number
@"And so am I!"
42
```

19
conventions.md
View File

@ -74,22 +74,25 @@ interior portions of a tree.
comments. Special syntax exists for such string annotations, though
the usual `@`-prefixed annotation notation can also be used.
;I am a comment for the Dictionary
# I am a comment for the Dictionary
{
;I am a comment for the key
key: ;I am a comment for the value
# I am a comment for the key
key: # I am a comment for the value
value
}
;I am a comment for this entire IOList
# I am a comment for this entire IOList, as are the next three lines.
#
# The previous line (containing only hash-newline) adds an empty
# string to the annotations attached to the entire IOList.
[
#x"00010203"
;I am a comment for the middle half of the IOList
;A second comment for the same portion of the IOList
@ ;I am the first and only comment for the following comment
# I am a comment for the middle half of the IOList
# A second comment for the same portion of the IOList
@ # I am the first and only comment for the following comment
"A third (itself commented!) comment for the same part of the IOList"
[
;"I am a comment for the following ByteString"
# I am a comment for the following ByteString
#x"04050607"
#x"08090A0B"
]

4
implementations/javascript/packages/core/src/reader.ts
View File

@ -336,7 +336,7 @@ export class Reader<T> {
case '|':
return Symbol.for(this.state.readString('|'));
case ';':
return this.annotateNextWith(this.readCommentLine());
this.state.error('Semicolon is reserved syntax', startPos);
case '@':
return this.annotateNextWith(this.next());
case ':':
@ -344,6 +344,8 @@ export class Reader<T> {
case '#': {
const c = this.state.nextchar();
switch (c) {
case ' ': case '\t': return this.annotateNextWith(this.readCommentLine());
case '\n': case '\r': return this.annotateNextWith('');
case 'f': return false;
case 't': return true;
case '{': return this.seq(new Set<T>(), (v, s) => s.add(v), '}');

44
implementations/python/preserves/text.py
View File

@ -165,20 +165,6 @@ class Parser(TextCodec):
break
self.skip()
def gather_annotations(self):
vs = []
while True:
self.skip_whitespace()
c = self.peek()
if c == ';':
self.skip()
vs.append(self.comment_line())
elif c == '@':
self.skip()
vs.append(self.next())
else:
return vs
def comment_line(self):
s = []
while True:
@ -317,6 +303,12 @@ class Parser(TextCodec):
def wrap(self, v):
return Annotated(v) if self.include_annotations else v
def unshift_annotation(self, a, v):
if self.include_annotations:
# TODO: this will end up O(n^2) for multiple annotations in a row
v.annotations.insert(0, a)
return v
def next(self):
"""Reads the next complete `Value` from the internal buffer, raising
[ShortPacket][preserves.error.ShortPacket] if too few bytes are available, or
@ -331,17 +323,18 @@ class Parser(TextCodec):
if c == '|':
self.skip()
return self.wrap(Symbol(self.read_string('|')))
if c in ';@':
annotations = self.gather_annotations()
v = self.next()
if self.include_annotations:
v.annotations = annotations + v.annotations
return v
if c == '@':
self.skip()
return self.unshift_annotation(self.next(), self.next())
if c == ';':
raise DecodeError('Semicolon is reserved syntax')
if c == ':':
raise DecodeError('Unexpected key/value separator between items')
if c == '#':
self.skip()
c = self.nextchar()
if c in ' \t': return self.unshift_annotation(self.comment_line(), self.next())
if c in '\n\r': return self.unshift_annotation('', self.next())
if c == 'f': return self.wrap(False)
if c == 't': return self.wrap(True)
if c == '{': return self.wrap(frozenset(self.upto('}')))
@ -353,17 +346,6 @@ class Parser(TextCodec):
if c == 'd': return self.wrap(self.read_hex_float(8))
raise DecodeError('Invalid #x syntax')
if c == '[': return self.wrap(self.read_base64_binary())
if c == '=':
old_ann = self.include_annotations
self.include_annotations = True
bs_val = self.next()
self.include_annotations = old_ann
if len(bs_val.annotations) > 0:
raise DecodeError('Annotations not permitted after #=')
bs_val = bs_val.item
if not isinstance(bs_val, bytes):
raise DecodeError('ByteString must follow #=')
return self.wrap(Decoder(bs_val, include_annotations = self.include_annotations).next())
if c == '!':
if self.parse_embedded is None:
raise DecodeError('No parse_embedded function supplied')

8
implementations/python/preserves/values.py
View File

@ -538,15 +538,15 @@ class Annotated(object):
```python
>>> import preserves
>>> a = preserves.parse('''
... ; A comment
... # A comment
... [1 2 3]
... ''', include_annotations=True)
>>> a
@' A comment' (1, 2, 3)
@'A comment' (1, 2, 3)
>>> a.item
(1, 2, 3)
>>> a.annotations
[' A comment']
['A comment']
>>> a == (1, 2, 3)
True
>>> a == preserves.parse('@xyz [1 2 3]', include_annotations=True)
@ -562,7 +562,7 @@ class Annotated(object):
>>> a.item[0].annotations
[]
>>> print(preserves.stringify(a))
@" A comment" [1 2 3]
@"A comment" [1 2 3]
>>> print(preserves.stringify(a, include_annotations=False))
[1 2 3]

2
implementations/python/tests/samples.pr
View File

@ -52,7 +52,7 @@
annotation5: <Test #x"85 B3026172 B4 B30152 85 B3026166 B30166 84" @ar <R @af f>>
annotation6: <Test #x"B4 85 B3026172 B30152 85 B3026166 B30166 84" <@ar R @af f>>
annotation7:
;Stop reading symbols at @ -- this test has three separate annotations
# Stop reading symbols at @ -- this test has three separate annotations
<Test #x"85 B30161 85 B30162 85 B30163 B584" @a@b@c[]>
bytes2: <Test #x"B20568656c6c6f" #"hello">
bytes2a: <Test @"Internal whitespace is allowed, including commas!" #x"B2, 05, 68, 65, 6c, 6c, 6f" #"hello">

42
implementations/racket/preserves/preserves-schema/schema.prs
View File

@ -1,7 +1,7 @@
@<EmacsMode "-*- preserves -*-">
; TODO: some kind of constants
; TODO: rename "version" to "schema-version" ?
# TODO: some kind of constants
# TODO: rename "version" to "schema-version" ?
version 1 .
@ -14,7 +14,7 @@ Schema = <schema {
definitions: Definitions
}>.
; version 1 .
# version 1 .
Version = 1 .
EmbeddedTypeName = #f / Ref .
@ -22,58 +22,58 @@ EmbeddedTypeName = #f / Ref .
Definitions = { symbol: Definition ...:... }.
Definition =
; Pattern / Pattern / ...
# Pattern / Pattern / ...
/ <or [@pattern0 NamedAlternative @pattern1 NamedAlternative @patternN NamedAlternative ...]>
; Pattern & Pattern & ...
# Pattern & Pattern & ...
/ <and [@pattern0 NamedPattern @pattern1 NamedPattern @patternN NamedPattern ...]>
; Pattern
# Pattern
/ Pattern
.
Pattern = SimplePattern / CompoundPattern .
SimplePattern =
; any
# any
/ =any
; special builtins: bool, float, double, int, string, bytes, symbol
# special builtins: bool, float, double, int, string, bytes, symbol
/ <atom @atomKind AtomKind>
; matches an embedded value in the input: #!p
# matches an embedded value in the input: #!p
/ <embedded @interface SimplePattern>
; =symbol, <<lit> any>, or plain non-symbol atom
# =symbol, <<lit> any>, or plain non-symbol atom
/ <lit @value any>
; [p ...] ----> <seqof <ref p>>; see also tuplePrefix below.
# [p ...] ----> <seqof <ref p>>; see also tuplePrefix below.
/ <seqof @pattern SimplePattern>
; #{p} ----> <setof <ref p>>
# #{p} ----> <setof <ref p>>
/ <setof @pattern SimplePattern>
; {k: v, ...:...} ----> <dictof <ref k> <ref v>>
# {k: v, ...:...} ----> <dictof <ref k> <ref v>>
/ <dictof @key SimplePattern @value SimplePattern>
; symbol, symbol.symbol, symbol.symbol.symbol, ...
# symbol, symbol.symbol, symbol.symbol.symbol, ...
/ Ref
.
CompoundPattern =
; <label a b c> ----> <rec <lit label> <tuple [<ref a> <ref b> <ref c>]>>
; except for record labels
; <<rec> x y> ---> <rec <ref x> <ref y>>
# <label a b c> ----> <rec <lit label> <tuple [<ref a> <ref b> <ref c>]>>
# except for record labels
# <<rec> x y> ---> <rec <ref x> <ref y>>
/ <rec @label NamedPattern @fields NamedPattern>
; [a b c] ----> <tuple [<ref a> <ref b> <ref c>]>
# [a b c] ----> <tuple [<ref a> <ref b> <ref c>]>
/ <tuple @patterns [NamedPattern ...]>
; [a b c ...] ----> <tuplePrefix [<ref a> <ref b>] <seqof <ref c>>>
; TODO: [@fixed0 NamedPattern @fixedN NamedPattern ...]
# [a b c ...] ----> <tuplePrefix [<ref a> <ref b>] <seqof <ref c>>>
# TODO: [@fixed0 NamedPattern @fixedN NamedPattern ...]
/ <tuplePrefix @fixed [NamedPattern ...] @variable NamedSimplePattern>
; {a: b, c: d} ----> <dict {a: <ref b>, c: <ref d>}>
# {a: b, c: d} ----> <dict {a: <ref b>, c: <ref d>}>
/ <dict @entries DictionaryEntries>
.

4
implementations/racket/preserves/preserves/read-text.rkt
View File

@ -71,12 +71,14 @@
[#\" (read-string #\")]
[(== PIPE) (string->symbol (read-string PIPE))]
[#\; (annotate-next-with (read-comment-line))]
[#\@ (annotate-next-with (next))]
[#\; (parse-error "Semicolon is reserved syntax")]
[#\: (parse-error "Unexpected key/value separator between items")]
[#\# (match (next-char)
[(or #\space #\tab) (annotate-next-with (read-comment-line))]
[(or #\newline #\return) (annotate-next-with "")]
[#\f #f]
[#\t #t]
[#\{ (sequence-fold (set) set-add* values #\})]

2
implementations/racket/preserves/preserves/tests/samples.pr
View File

@ -52,7 +52,7 @@
annotation5: <Test #x"85 B3026172 B4 B30152 85 B3026166 B30166 84" @ar <R @af f>>
annotation6: <Test #x"B4 85 B3026172 B30152 85 B3026166 B30166 84" <@ar R @af f>>
annotation7:
;Stop reading symbols at @ -- this test has three separate annotations
# Stop reading symbols at @ -- this test has three separate annotations
<Test #x"85 B30161 85 B30162 85 B30163 B584" @a@b@c[]>
bytes2: <Test #x"B20568656c6c6f" #"hello">
bytes2a: <Test @"Internal whitespace is allowed, including commas!" #x"B2, 05, 68, 65, 6c, 6c, 6f" #"hello">

207
implementations/rust/preserves/src/value/text/reader.rs
View File

@ -99,31 +99,52 @@ impl<'de, 'src, D: Embeddable, Dec: DomainParse<D>, S: BinarySource<'de>>
}
}
fn gather_annotations<N: NestedValue<Embedded = D>>(&mut self) -> ReaderResult<Vec<N>> {
let mut vs = Vec::new();
fn gather_annotations<N: NestedValue<Embedded = D>>(&mut self, vs: &mut Vec<N>) -> ReaderResult<()> {
loop {
self.skip_whitespace();
match self.peek()? {
b';' => {
b'#' => {
let m = self.source.mark()?;
self.skip()?;
vs.push(N::new(self.comment_line()?))
match self.next_byte()? {
b' ' | b'\t' => vs.push(N::new(self.comment_line()?)),
b'\n' | b'\r' => vs.push(N::new("")),
_ => {
self.source.restore(&m)?;
return Ok(());
}
}
}
b'@' => {
self.skip()?;
vs.push(self.demand_next(true)?)
}
_ => return Ok(vs),
_ => return Ok(()),
}
}
}
fn prepend_annotations_to_next<N: NestedValue<Embedded = D>>(&mut self, mut annotations: Vec<N>) -> ReaderResult<N> {
let (existing_annotations, v) = Reader::<N>::demand_next(self, true)?.pieces();
annotations.extend_from_slice(existing_annotations.slice());
Ok(N::wrap(Annotations::new(Some(annotations)), v))
}
fn skip_annotations(&mut self) -> ReaderResult<()> {
loop {
self.skip_whitespace();
match self.peek()? {
b';' => {
b'#' => {
let m = self.source.mark()?;
self.skip()?;
self.comment_line()?;
match self.next_byte()? {
b' ' | b'\t' => { self.comment_line()?; () }
b'\n' | b'\r' => (),
_ => {
self.source.restore(&m)?;
return Ok(());
}
}
}
b'@' => {
self.skip()?;
@ -395,87 +416,102 @@ impl<'de, 'src, N: NestedValue, Dec: DomainParse<N::Embedded>, S: BinarySource<'
for TextReader<'de, 'src, N::Embedded, Dec, S>
{
fn next(&mut self, read_annotations: bool) -> io::Result<Option<N>> {
self.skip_whitespace();
let c = match self.peek() {
Ok(c) => c,
Err(e) if is_eof_io_error(&e) => return Ok(None),
Err(e) => return Err(e.into()),
};
Ok(Some(match c {
b'"' => {
self.skip()?;
N::new(self.read_string(b'"')?)
}
b'|' => {
self.skip()?;
N::symbol(&self.read_string(b'|')?)
}
b';' | b'@' => {
if read_annotations {
let mut annotations = self.gather_annotations()?;
let (existing_annotations, v) =
Reader::<N>::demand_next(self, read_annotations)?.pieces();
annotations.extend_from_slice(existing_annotations.slice());
N::wrap(Annotations::new(Some(annotations)), v)
} else {
self.skip_annotations()?;
self.demand_next(read_annotations)?
'restart: loop {
self.skip_whitespace();
let c = match self.peek() {
Ok(c) => c,
Err(e) if is_eof_io_error(&e) => return Ok(None),
Err(e) => return Err(e.into()),
};
return Ok(Some(match c {
b'"' => {
self.skip()?;
N::new(self.read_string(b'"')?)
}
}
b':' => {
return Err(io_syntax_error(
"Unexpected key/value separator between items",
));
}
b'#' => {
self.skip()?;
match self.next_byte()? {
b'f' => N::new(false),
b't' => N::new(true),
b'{' => N::new(Set::from_iter(
self.upto(b'}', read_annotations)?.into_iter(),
)),
b'"' => self.read_literal_binary()?,
b'x' => match self.next_byte()? {
b'"' => N::new(&self.read_hex_binary()?[..]),
b'f' => self.read_hex_float(4)?,
b'd' => self.read_hex_float(8)?,
_ => return Err(io_syntax_error("Invalid #x syntax")),
},
b'[' => self.read_base64_binary()?,
b'!' => {
let v = self.next_iovalue(read_annotations)?;
Value::Embedded(self.dec.parse_embedded(&v)?).wrap()
}
other => {
return Err(io_syntax_error(&format!("Invalid # syntax: {:?}", other)))
b'|' => {
self.skip()?;
N::symbol(&self.read_string(b'|')?)
}
b';' => {
return Err(io_syntax_error(
"Semicolon is reserved syntax"
));
}
b'@' => {
if read_annotations {
let mut annotations = Vec::new();
self.gather_annotations(&mut annotations)?;
self.prepend_annotations_to_next(annotations)?
} else {
self.skip_annotations()?;
self.demand_next(read_annotations)?
}
}
}
b'<' => {
self.skip()?;
let vs = self.upto(b'>', read_annotations)?;
if vs.is_empty() {
return Err(io_syntax_error("Missing record label"));
b':' => {
return Err(io_syntax_error(
"Unexpected key/value separator between items",
));
}
Value::Record(Record(vs)).wrap()
}
b'[' => {
self.skip()?;
N::new(self.upto(b']', read_annotations)?)
}
b'{' => {
self.skip()?;
self.read_dictionary(read_annotations)?
}
b'>' => return Err(io_syntax_error("Unexpected >")),
b']' => return Err(io_syntax_error("Unexpected ]")),
b'}' => return Err(io_syntax_error("Unexpected }")),
other => {
self.skip()?;
self.read_raw_symbol_or_number(vec![other])?
}
}))
b'#' => {
self.skip()?;
match self.next_byte()? {
b' ' | b'\t' => {
if read_annotations {
let mut annotations = vec![N::new(self.comment_line()?)];
self.gather_annotations(&mut annotations)?;
self.prepend_annotations_to_next(annotations)?
} else {
self.comment_line()?;
continue 'restart;
}
}
b'f' => N::new(false),
b't' => N::new(true),
b'{' => N::new(Set::from_iter(
self.upto(b'}', read_annotations)?.into_iter(),
)),
b'"' => self.read_literal_binary()?,
b'x' => match self.next_byte()? {
b'"' => N::new(&self.read_hex_binary()?[..]),
b'f' => self.read_hex_float(4)?,
b'd' => self.read_hex_float(8)?,
_ => return Err(io_syntax_error("Invalid #x syntax")),
},
b'[' => self.read_base64_binary()?,
b'!' => {
let v = self.next_iovalue(read_annotations)?;
Value::Embedded(self.dec.parse_embedded(&v)?).wrap()
}
other => {
return Err(io_syntax_error(&format!("Invalid # syntax: {:?}", other)))
}
}
}
b'<' => {
self.skip()?;
let vs = self.upto(b'>', read_annotations)?;
if vs.is_empty() {
return Err(io_syntax_error("Missing record label"));
}
Value::Record(Record(vs)).wrap()
}
b'[' => {
self.skip()?;
N::new(self.upto(b']', read_annotations)?)
}
b'{' => {
self.skip()?;
self.read_dictionary(read_annotations)?
}
b'>' => return Err(io_syntax_error("Unexpected >")),
b']' => return Err(io_syntax_error("Unexpected ]")),
b'}' => return Err(io_syntax_error("Unexpected }")),
other => {
self.skip()?;
self.read_raw_symbol_or_number(vec![other])?
}
}))
}
}
fn open_record(&mut self, arity: Option<usize>) -> ReaderResult<B::Type> {
@ -625,7 +661,8 @@ impl<'de, 'src, N: NestedValue, Dec: DomainParse<N::Embedded>, S: BinarySource<'
}
fn next_annotations_and_token(&mut self) -> io::Result<(Vec<N>, Token<N>)> {
let annotations = self.gather_annotations()?;
let mut annotations = Vec::new();
self.gather_annotations(&mut annotations)?;
Ok((annotations, self.next_token(true)?))
}
}

56
preserves-path.md
View File

@ -23,7 +23,7 @@ dicts, but also for sets).
A sequence of steps, applied one after the other, flatmap-style.
step ... ;; Applies steps one after the other, flatmap-style
step ... # Applies steps one after the other, flatmap-style
Each step transforms an input document into zero or more related
documents. A step is an axis or a filter.
@ -37,27 +37,27 @@ etc.
Precedence groupings from highest to lowest. Within a grouping, no
mixed precedence is permitted.
selector ;; Applies steps one after the other, flatmap-style
selector # Applies steps one after the other, flatmap-style
! pred ;; "not" of a predicate
! pred # "not" of a predicate
pred + pred + ... ;; "or" of predicates
pred & pred & ... ;; "and" of predicates
pred + pred + ... # "or" of predicates
pred & pred & ... # "and" of predicates
## Axes
Axes: move around, applying filters after moving
/ ;; Moves into immediate children (values / fields)
// ;; Flattens children recursively
. key ;; Moves into named child
.^ ;; Moves into record label
.keys ;; Moves into *keys* rather than values
.length ;; Moves into the number of keys
.annotations ;; Moves into any annotations that might be present
.embedded ;; Moves into the representation of an embedded value
% name ;; Moves into successful Preserves Schema parse of definition `name`
%- name ;; Moves into successful Preserves Schema unparse of definition `name`
/ # Moves into immediate children (values / fields)
// # Flattens children recursively
. key # Moves into named child
.^ # Moves into record label
.keys # Moves into *keys* rather than values
.length # Moves into the number of keys
.annotations # Moves into any annotations that might be present
.embedded # Moves into the representation of an embedded value
% name # Moves into successful Preserves Schema parse of definition `name`
%- name # Moves into successful Preserves Schema unparse of definition `name`
Sets have children, but no keys/length; Strings, ByteStrings and
Symbols have no children, but have keys/length.
@ -66,10 +66,10 @@ Symbols have no children, but have keys/length.
Filters: narrow down a selection without moving
* ;; Accepts all
[!] ;; Rejects all (just a use of `[pred]`)
* # Accepts all
[!] # Rejects all (just a use of `[pred]`)
eq literal ;; Matches values (equal to/less than/greater than/etc.) the literal
eq literal # Matches values (equal to/less than/greater than/etc.) the literal
= literal
ne literal
!= literal
@ -78,21 +78,21 @@ Filters: narrow down a selection without moving
le literal
ge literal
re regex ;; Matches strings and symbols by POSIX extended regular expression
re regex # Matches strings and symbols by POSIX extended regular expression
=r regex
[pred] ;; Applies predicate to each input; keeps inputs yielding truth
[pred] # Applies predicate to each input; keeps inputs yielding truth
^ literal ;; Matches a record having a the literal as its label -- equivalent to [.^ = literal]
^ literal # Matches a record having a the literal as its label -- equivalent to [.^ = literal]
~real ;; Promotes int and float to double, passes on double unchanged, rejects others
;; Out-of-range ints (too big or too small) become various double infinities
;; Converting high-magnitude ints causes loss of precision
~real # Promotes int and float to double, passes on double unchanged, rejects others
# Out-of-range ints (too big or too small) become various double infinities
# Converting high-magnitude ints causes loss of precision
~int ;; Converts float and double to closest integer, where possible
;; NaN and infinities are rejected
~int # Converts float and double to closest integer, where possible
# NaN and infinities are rejected
bool ;; Type filters
bool # Type filters
float
double
int
@ -122,7 +122,7 @@ Design choice: How should comparison work? Should `lt 1.0f` accept not only `0.9
## Functions
<count selector> ;; Counts number of results of selector
<count selector> # Counts number of results of selector
## Transformers

36
preserves-schema.md
View File

@ -544,7 +544,7 @@ A `Bundle` collects a number of `Schema`s, each named by a
A `Version` names the version of the schema language in use. At
present, it must be `1`.
; version 1 .
# version 1 .
Version = 1 .
An `EmbeddedTypeName` specifies the type of embedded values within
@ -560,17 +560,17 @@ ensure that each `or` or `and` record has at least two members.
Definitions = { symbol: Definition ...:... }.
Definition =
; Pattern / Pattern / ...
# Pattern / Pattern / ...
/ <or [@pattern0 NamedAlternative
@pattern1 NamedAlternative
@patternN NamedAlternative ...]>
; Pattern & Pattern & ...
# Pattern & Pattern & ...
/ <and [@pattern0 NamedPattern
@pattern1 NamedPattern
@patternN NamedPattern ...]>
; Pattern
# Pattern
/ Pattern
.
@ -583,28 +583,28 @@ Each `Pattern` is either a simple or compound pattern:
Simple patterns are as described above:
SimplePattern =
; any
# any
/ =any
; special builtins: bool, float, double, int, string, bytes, symbol
# special builtins: bool, float, double, int, string, bytes, symbol
/ <atom @atomKind AtomKind>
; matches an embedded value in the input: #!p
# matches an embedded value in the input: #!p
/ <embedded @interface SimplePattern>
; =symbol, <<lit> any>, or plain non-symbol atom
# =symbol, <<lit> any>, or plain non-symbol atom
/ <lit @value any>
; [p ...] ----> <seqof <ref p>>; see also tuplePrefix below.
# [p ...] ----> <seqof <ref p>># see also tuplePrefix below.
/ <seqof @pattern SimplePattern>
; #{p} ----> <setof <ref p>>
# #{p} ----> <setof <ref p>>
/ <setof @pattern SimplePattern>
; {k: v, ...:...} ----> <dictof <ref k> <ref v>>
# {k: v, ...:...} ----> <dictof <ref k> <ref v>>
/ <dictof @key SimplePattern @value SimplePattern>
; symbol, symbol.symbol, symbol.symbol.symbol, ...
# symbol, symbol.symbol, symbol.symbol.symbol, ...
/ Ref
.
@ -619,18 +619,18 @@ Simple patterns are as described above:
Compound patterns involve optionally-named subpatterns:
CompoundPattern =
; <label a b c> ----> <rec <lit label> <tuple [<ref a> <ref b> <ref c>]>>
; except for record labels
; <<rec> x y> ---> <rec <ref x> <ref y>>
# <label a b c> ----> <rec <lit label> <tuple [<ref a> <ref b> <ref c>]>>
# except for record labels
# <<rec> x y> ---> <rec <ref x> <ref y>>
/ <rec @label NamedPattern @fields NamedPattern>
; [a b c] ----> <tuple [<ref a> <ref b> <ref c>]>
# [a b c] ----> <tuple [<ref a> <ref b> <ref c>]>
/ <tuple @patterns [NamedPattern ...]>
; [a b c ...] ----> <tuplePrefix [<ref a> <ref b>] <seqof <ref c>>>
# [a b c ...] ----> <tuplePrefix [<ref a> <ref b>] <seqof <ref c>>>
/ <tuplePrefix @fixed [NamedPattern ...] @variable NamedSimplePattern>
; {a: b, c: d} ----> <dict {a: <ref b>, c: <ref d>}>
# {a: b, c: d} ----> <dict {a: <ref b>, c: <ref d>}>
/ <dict @entries DictionaryEntries>
.

9
preserves-text.md
View File

@ -268,11 +268,14 @@ named “`Value`” without altering the semantic class of `Value`s.
interpreted as comments associated with that value. Comments are
sufficiently common that special syntax exists for them.
Value =/ ws ";" linecomment (CR / LF) Value
Value =/ ws ("#" [(%x20 / %x09) linecomment]) (CR / LF) Value
linecomment = *<any unicode scalar value except CR or LF>
When written this way, everything between the `;` and the end of the line
is included in the string annotating the `Value`.
When written this way, everything between the hash-space or hash-tab and
the end of the line is included in the string annotating the `Value`.
Comments that are just hash `#` followed immediately by newline yield an
empty-string annotation.
**Equivalence.** Annotations appear within syntax denoting a `Value`;
however, the annotations are not part of the denoted value. They are

14
preserves.el
View File

@ -33,13 +33,15 @@
"Syntax table in use in preserves-mode buffers.")
;; (modify-syntax-entry ?' "\"" preserves-mode-syntax-table)
(modify-syntax-entry ?\n ">" preserves-mode-syntax-table)
(modify-syntax-entry ?\r ">" preserves-mode-syntax-table)
(modify-syntax-entry ?\; "<" preserves-mode-syntax-table)
(modify-syntax-entry ?\n "> 2" preserves-mode-syntax-table)
(modify-syntax-entry ?\r "> 2" preserves-mode-syntax-table)
(modify-syntax-entry ?\t " 2" preserves-mode-syntax-table)
(modify-syntax-entry ?< "(>" preserves-mode-syntax-table)
(modify-syntax-entry ?> ")<" preserves-mode-syntax-table)
(modify-syntax-entry ?# "' 1" preserves-mode-syntax-table)
(modify-syntax-entry ? " 2" preserves-mode-syntax-table)
(mapcar #'(lambda (x) (modify-syntax-entry x "_" preserves-mode-syntax-table))
'(?- ?_ ?$ ?? ?! ?* ?+ ?~ ?: ?= ?|))
'(?- ?_ ?$ ?? ?! ?* ?+ ?~ ?: ?= ?| ?\;))
(mapcar #'(lambda (x) (modify-syntax-entry x "." preserves-mode-syntax-table))
'(?.))
@ -55,9 +57,9 @@
(make-local-variable 'comment-end)
(make-local-variable 'comment-start-skip)
(setq comment-use-syntax t)
(setq comment-start ";")
(setq comment-start "# ")
(setq comment-end "")
(setq comment-start-skip "; *")
(setq comment-start-skip "# +")
(make-local-variable 'font-lock-defaults)
(setq font-lock-defaults '(preserves-font-lock-keywords nil nil ()))
(make-local-variable 'indent-line-function)

2
python/0.990.0/search/search_index.json
View File

File diff suppressed because one or more lines are too long

176
python/0.990.0/text/index.html
View File

@ -813,21 +813,21 @@ omitted.</p></td>
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">475</span>
<span class="normal">476</span>
<span class="normal">477</span>
<span class="normal">478</span>
<span class="normal">479</span>
<span class="normal">480</span>
<span class="normal">481</span>
<span class="normal">482</span>
<span class="normal">483</span>
<span class="normal">484</span>
<span class="normal">485</span>
<span class="normal">486</span>
<span class="normal">487</span>
<span class="normal">488</span>
<span class="normal">489</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">457</span>
<span class="normal">458</span>
<span class="normal">459</span>
<span class="normal">460</span>
<span class="normal">461</span>
<span class="normal">462</span>
<span class="normal">463</span>
<span class="normal">464</span>
<span class="normal">465</span>
<span class="normal">466</span>
<span class="normal">467</span>
<span class="normal">468</span>
<span class="normal">469</span>
<span class="normal">470</span>
<span class="normal">471</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span>
<span class="n">format_embedded</span><span class="o">=</span><span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">x</span><span class="p">,</span>
<span class="n">indent</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
<span class="n">with_commas</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
@ -874,16 +874,16 @@ representation of <code>v</code>.</p>
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">545</span>
<span class="normal">546</span>
<span class="normal">547</span>
<span class="normal">548</span>
<span class="normal">549</span>
<span class="normal">550</span>
<span class="normal">551</span>
<span class="normal">552</span>
<span class="normal">553</span>
<span class="normal">554</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">v</span><span class="p">):</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">527</span>
<span class="normal">528</span>
<span class="normal">529</span>
<span class="normal">530</span>
<span class="normal">531</span>
<span class="normal">532</span>
<span class="normal">533</span>
<span class="normal">534</span>
<span class="normal">535</span>
<span class="normal">536</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">v</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Extend `self.chunks` with at least one chunk, together making up the text</span>
<span class="sd"> representation of `v`.&quot;&quot;&quot;</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">chunks</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">nesting</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
@ -915,9 +915,9 @@ representation of <code>v</code>.</p>
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">496</span>
<span class="normal">497</span>
<span class="normal">498</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">contents</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">478</span>
<span class="normal">479</span>
<span class="normal">480</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">contents</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Returns a `str` constructed from the join of the chunks in `self.chunks`.&quot;&quot;&quot;</span>
<span class="k">return</span> <span class="sa">u</span><span class="s1">&#39;&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">chunks</span><span class="p">)</span>
</code></pre></div></td></tr></table></div>
@ -943,10 +943,10 @@ indenting mode.</p>
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">500</span>
<span class="normal">501</span>
<span class="normal">502</span>
<span class="normal">503</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">is_indenting</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">482</span>
<span class="normal">483</span>
<span class="normal">484</span>
<span class="normal">485</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">is_indenting</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Returns `True` iff this [Formatter][preserves.text.Formatter] is in pretty-printing</span>
<span class="sd"> indenting mode.&quot;&quot;&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">indent_delta</span> <span class="o">&gt;</span> <span class="mi">0</span>
@ -1187,7 +1187,15 @@ text from the front of <code>self.input_buffer</code> and resetting <code>self.i
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">320</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">312</span>
<span class="normal">313</span>
<span class="normal">314</span>
<span class="normal">315</span>
<span class="normal">316</span>
<span class="normal">317</span>
<span class="normal">318</span>
<span class="normal">319</span>
<span class="normal">320</span>
<span class="normal">321</span>
<span class="normal">322</span>
<span class="normal">323</span>
@ -1236,25 +1244,7 @@ text from the front of <code>self.input_buffer</code> and resetting <code>self.i
<span class="normal">366</span>
<span class="normal">367</span>
<span class="normal">368</span>
<span class="normal">369</span>
<span class="normal">370</span>
<span class="normal">371</span>
<span class="normal">372</span>
<span class="normal">373</span>
<span class="normal">374</span>
<span class="normal">375</span>
<span class="normal">376</span>
<span class="normal">377</span>
<span class="normal">378</span>
<span class="normal">379</span>
<span class="normal">380</span>
<span class="normal">381</span>
<span class="normal">382</span>
<span class="normal">383</span>
<span class="normal">384</span>
<span class="normal">385</span>
<span class="normal">386</span>
<span class="normal">387</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">next</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="normal">369</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">next</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Reads the next complete `Value` from the internal buffer, raising</span>
<span class="sd"> [ShortPacket][preserves.error.ShortPacket] if too few bytes are available, or</span>
<span class="sd"> [DecodeError][preserves.error.DecodeError] if the input is invalid somehow.</span>
@ -1268,17 +1258,18 @@ text from the front of <code>self.input_buffer</code> and resetting <code>self.i
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;|&#39;</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">skip</span><span class="p">()</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="n">Symbol</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">read_string</span><span class="p">(</span><span class="s1">&#39;|&#39;</span><span class="p">)))</span>
<span class="k">if</span> <span class="n">c</span> <span class="ow">in</span> <span class="s1">&#39;;@&#39;</span><span class="p">:</span>
<span class="n">annotations</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">gather_annotations</span><span class="p">()</span>
<span class="n">v</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">next</span><span class="p">()</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">include_annotations</span><span class="p">:</span>
<span class="n">v</span><span class="o">.</span><span class="n">annotations</span> <span class="o">=</span> <span class="n">annotations</span> <span class="o">+</span> <span class="n">v</span><span class="o">.</span><span class="n">annotations</span>
<span class="k">return</span> <span class="n">v</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;@&#39;</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">skip</span><span class="p">()</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">unshift_annotation</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">next</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">next</span><span class="p">())</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;;&#39;</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">DecodeError</span><span class="p">(</span><span class="s1">&#39;Semicolon is reserved syntax&#39;</span><span class="p">)</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;:&#39;</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">DecodeError</span><span class="p">(</span><span class="s1">&#39;Unexpected key/value separator between items&#39;</span><span class="p">)</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;#&#39;</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">skip</span><span class="p">()</span>
<span class="n">c</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">nextchar</span><span class="p">()</span>
<span class="k">if</span> <span class="n">c</span> <span class="ow">in</span> <span class="s1">&#39; </span><span class="se">\t</span><span class="s1">&#39;</span><span class="p">:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">unshift_annotation</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">comment_line</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">next</span><span class="p">())</span>
<span class="k">if</span> <span class="n">c</span> <span class="ow">in</span> <span class="s1">&#39;</span><span class="se">\n\r</span><span class="s1">&#39;</span><span class="p">:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">unshift_annotation</span><span class="p">(</span><span class="s1">&#39;&#39;</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">next</span><span class="p">())</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;f&#39;</span><span class="p">:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="kc">False</span><span class="p">)</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;t&#39;</span><span class="p">:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;{&#39;</span><span class="p">:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="nb">frozenset</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">upto</span><span class="p">(</span><span class="s1">&#39;}&#39;</span><span class="p">)))</span>
@ -1290,17 +1281,6 @@ text from the front of <code>self.input_buffer</code> and resetting <code>self.i
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;d&#39;</span><span class="p">:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">read_hex_float</span><span class="p">(</span><span class="mi">8</span><span class="p">))</span>
<span class="k">raise</span> <span class="n">DecodeError</span><span class="p">(</span><span class="s1">&#39;Invalid #x syntax&#39;</span><span class="p">)</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;[&#39;</span><span class="p">:</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">read_base64_binary</span><span class="p">())</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;=&#39;</span><span class="p">:</span>
<span class="n">old_ann</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">include_annotations</span>
<span class="bp">self</span><span class="o">.</span><span class="n">include_annotations</span> <span class="o">=</span> <span class="kc">True</span>
<span class="n">bs_val</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">next</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">include_annotations</span> <span class="o">=</span> <span class="n">old_ann</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">bs_val</span><span class="o">.</span><span class="n">annotations</span><span class="p">)</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">DecodeError</span><span class="p">(</span><span class="s1">&#39;Annotations not permitted after #=&#39;</span><span class="p">)</span>
<span class="n">bs_val</span> <span class="o">=</span> <span class="n">bs_val</span><span class="o">.</span><span class="n">item</span>
<span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">bs_val</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">):</span>
<span class="k">raise</span> <span class="n">DecodeError</span><span class="p">(</span><span class="s1">&#39;ByteString must follow #=&#39;</span><span class="p">)</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="n">Decoder</span><span class="p">(</span><span class="n">bs_val</span><span class="p">,</span> <span class="n">include_annotations</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">include_annotations</span><span class="p">)</span><span class="o">.</span><span class="n">next</span><span class="p">())</span>
<span class="k">if</span> <span class="n">c</span> <span class="o">==</span> <span class="s1">&#39;!&#39;</span><span class="p">:</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">parse_embedded</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">DecodeError</span><span class="p">(</span><span class="s1">&#39;No parse_embedded function supplied&#39;</span><span class="p">)</span>
@ -1345,15 +1325,15 @@ text from the front of <code>self.input_buffer</code> and resetting <code>self.i
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">389</span>
<span class="normal">390</span>
<span class="normal">391</span>
<span class="normal">392</span>
<span class="normal">393</span>
<span class="normal">394</span>
<span class="normal">395</span>
<span class="normal">396</span>
<span class="normal">397</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">try_next</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">371</span>
<span class="normal">372</span>
<span class="normal">373</span>
<span class="normal">374</span>
<span class="normal">375</span>
<span class="normal">376</span>
<span class="normal">377</span>
<span class="normal">378</span>
<span class="normal">379</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">try_next</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Like [next][preserves.text.Parser.next], but returns `None` instead of raising</span>
<span class="sd"> [ShortPacket][preserves.error.ShortPacket].&quot;&quot;&quot;</span>
<span class="n">start</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span>
@ -1419,16 +1399,16 @@ text from the front of <code>self.input_buffer</code> and resetting <code>self.i
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">408</span>
<span class="normal">409</span>
<span class="normal">410</span>
<span class="normal">411</span>
<span class="normal">412</span>
<span class="normal">413</span>
<span class="normal">414</span>
<span class="normal">415</span>
<span class="normal">416</span>
<span class="normal">417</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">parse</span><span class="p">(</span><span class="n">text</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">390</span>
<span class="normal">391</span>
<span class="normal">392</span>
<span class="normal">393</span>
<span class="normal">394</span>
<span class="normal">395</span>
<span class="normal">396</span>
<span class="normal">397</span>
<span class="normal">398</span>
<span class="normal">399</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">parse</span><span class="p">(</span><span class="n">text</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Yields the first complete encoded value from `text`, passing `kwargs` through to the</span>
<span class="sd"> [Parser][preserves.text.Parser] constructor. Raises exceptions as per</span>
<span class="sd"> [next][preserves.text.Parser.next].</span>
@ -1461,10 +1441,10 @@ text from the front of <code>self.input_buffer</code> and resetting <code>self.i
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">419</span>
<span class="normal">420</span>
<span class="normal">421</span>
<span class="normal">422</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">parse_with_annotations</span><span class="p">(</span><span class="n">bs</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">401</span>
<span class="normal">402</span>
<span class="normal">403</span>
<span class="normal">404</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">parse_with_annotations</span><span class="p">(</span><span class="n">bs</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Like [parse][preserves.text.parse], but supplying `include_annotations=True` to the</span>
<span class="sd"> [Parser][preserves.text.Parser] constructor.&quot;&quot;&quot;</span>
<span class="k">return</span> <span class="n">Parser</span><span class="p">(</span><span class="n">input_buffer</span><span class="o">=</span><span class="n">bs</span><span class="p">,</span> <span class="n">include_annotations</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span><span class="o">.</span><span class="n">next</span><span class="p">()</span>
@ -1491,12 +1471,12 @@ underlying <a class="autorefs autorefs-internal" href="#preserves.text.Formatter
<details class="quote">
<summary>Source code in <code>preserves/text.py</code></summary>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">606</span>
<span class="normal">607</span>
<span class="normal">608</span>
<span class="normal">609</span>
<span class="normal">610</span>
<span class="normal">611</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">stringify</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal">588</span>
<span class="normal">589</span>
<span class="normal">590</span>
<span class="normal">591</span>
<span class="normal">592</span>
<span class="normal">593</span></pre></div></td><td class="code"><div><pre><span></span><code><span class="k">def</span> <span class="nf">stringify</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Convert a single `Value` `v` to a string. Any supplied `kwargs` are passed on to the</span>
<span class="sd"> underlying [Formatter][preserves.text.Formatter] constructor.&quot;&quot;&quot;</span>
<span class="n">e</span> <span class="o">=</span> <span class="n">Formatter</span><span class="p">(</span><span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>

8
python/0.990.0/values/index.html
View File

@ -845,15 +845,15 @@ the underlying <code>Value</code>, ignoring the annotations. See the <a href="ht
about annotations</a>.</p>
<div class="highlight"><pre><span></span><code><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">preserves</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">a</span> <span class="o">=</span> <span class="n">preserves</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="s1">&#39;&#39;&#39;</span>
<span class="s1">... ; A comment</span>
<span class="s1">... # A comment</span>
<span class="s1">... [1 2 3]</span>
<span class="s1">... &#39;&#39;&#39;</span><span class="p">,</span> <span class="n">include_annotations</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">a</span>
<span class="o">@</span><span class="s1">&#39; A comment&#39;</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="o">@</span><span class="s1">&#39;A comment&#39;</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">a</span><span class="o">.</span><span class="n">item</span>
<span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">a</span><span class="o">.</span><span class="n">annotations</span>
<span class="p">[</span><span class="s1">&#39; A comment&#39;</span><span class="p">]</span>
<span class="p">[</span><span class="s1">&#39;A comment&#39;</span><span class="p">]</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">a</span> <span class="o">==</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="kc">True</span>
<span class="o">&gt;&gt;&gt;</span> <span class="n">a</span> <span class="o">==</span> <span class="n">preserves</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="s1">&#39;@xyz [1 2 3]&#39;</span><span class="p">,</span> <span class="n">include_annotations</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
@ -869,7 +869,7 @@ about annotations</a>.</p>
<span class="o">&gt;&gt;&gt;</span> <span class="n">a</span><span class="o">.</span><span class="n">item</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">annotations</span>
<span class="p">[]</span>
<span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">preserves</span><span class="o">.</span><span class="n">stringify</span><span class="p">(</span><span class="n">a</span><span class="p">))</span>
<span class="o">@</span><span class="s2">&quot; A comment&quot;</span> <span class="p">[</span><span class="mi">1</span> <span class="mi">2</span> <span class="mi">3</span><span class="p">]</span>
<span class="o">@</span><span class="s2">&quot;A comment&quot;</span> <span class="p">[</span><span class="mi">1</span> <span class="mi">2</span> <span class="mi">3</span><span class="p">]</span>
<span class="o">&gt;&gt;&gt;</span> <span class="nb">print</span><span class="p">(</span><span class="n">preserves</span><span class="o">.</span><span class="n">stringify</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">include_annotations</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
<span class="p">[</span><span class="mi">1</span> <span class="mi">2</span> <span class="mi">3</span><span class="p">]</span>
</code></pre></div>

3
questions.md
View File

@ -6,7 +6,8 @@ Q. Should "symbols" instead be URIs? Relative, usually; relative to
what? Some domain-specific base URI?
Q. Literal small integers: are they pulling their weight? They're not
absolutely necessary.
absolutely necessary. A. No, they have been removed (as part of the changes
at version 0.990).
Q. Should we go for trying to make the data ordering line up with the
encoding ordering? We'd have to only use streaming forms, and avoid

42
schema/schema.prs
View File

@ -1,7 +1,7 @@
@<EmacsMode "-*- preserves -*-">
; TODO: some kind of constants
; TODO: rename "version" to "schema-version" ?
# TODO: some kind of constants
# TODO: rename "version" to "schema-version" ?
version 1 .
@ -14,7 +14,7 @@ Schema = <schema {
definitions: Definitions
}>.
; version 1 .
# version 1 .
Version = 1 .
EmbeddedTypeName = #f / Ref .
@ -22,58 +22,58 @@ EmbeddedTypeName = #f / Ref .
Definitions = { symbol: Definition ...:... }.
Definition =
; Pattern / Pattern / ...
# Pattern / Pattern / ...
/ <or [@pattern0 NamedAlternative @pattern1 NamedAlternative @patternN NamedAlternative ...]>
; Pattern & Pattern & ...
# Pattern & Pattern & ...
/ <and [@pattern0 NamedPattern @pattern1 NamedPattern @patternN NamedPattern ...]>
; Pattern
# Pattern
/ Pattern
.
Pattern = SimplePattern / CompoundPattern .
SimplePattern =
; any
# any
/ =any
; special builtins: bool, float, double, int, string, bytes, symbol
# special builtins: bool, float, double, int, string, bytes, symbol
/ <atom @atomKind AtomKind>
; matches an embedded value in the input: #!p
# matches an embedded value in the input: #!p
/ <embedded @interface SimplePattern>
; =symbol, <<lit> any>, or plain non-symbol atom
# =symbol, <<lit> any>, or plain non-symbol atom
/ <lit @value any>
; [p ...] ----> <seqof <ref p>>; see also tuplePrefix below.
# [p ...] ----> <seqof <ref p>>; see also tuplePrefix below.
/ <seqof @pattern SimplePattern>
; #{p} ----> <setof <ref p>>
# #{p} ----> <setof <ref p>>
/ <setof @pattern SimplePattern>
; {k: v, ...:...} ----> <dictof <ref k> <ref v>>
# {k: v, ...:...} ----> <dictof <ref k> <ref v>>
/ <dictof @key SimplePattern @value SimplePattern>
; symbol, symbol.symbol, symbol.symbol.symbol, ...
# symbol, symbol.symbol, symbol.symbol.symbol, ...
/ Ref
.
CompoundPattern =
; <label a b c> ----> <rec <lit label> <tuple [<ref a> <ref b> <ref c>]>>
; except for record labels
; <<rec> x y> ---> <rec <ref x> <ref y>>
# <label a b c> ----> <rec <lit label> <tuple [<ref a> <ref b> <ref c>]>>
# except for record labels
# <<rec> x y> ---> <rec <ref x> <ref y>>
/ <rec @label NamedPattern @fields NamedPattern>
; [a b c] ----> <tuple [<ref a> <ref b> <ref c>]>
# [a b c] ----> <tuple [<ref a> <ref b> <ref c>]>
/ <tuple @patterns [NamedPattern ...]>
; [a b c ...] ----> <tuplePrefix [<ref a> <ref b>] <seqof <ref c>>>
; TODO: [@fixed0 NamedPattern @fixedN NamedPattern ...]
# [a b c ...] ----> <tuplePrefix [<ref a> <ref b>] <seqof <ref c>>>
# TODO: [@fixed0 NamedPattern @fixedN NamedPattern ...]
/ <tuplePrefix @fixed [NamedPattern ...] @variable NamedSimplePattern>
; {a: b, c: d} ----> <dict {a: <ref b>, c: <ref d>}>
# {a: b, c: d} ----> <dict {a: <ref b>, c: <ref d>}>
/ <dict @entries DictionaryEntries>
.

2
tests/samples.pr
View File

@ -52,7 +52,7 @@
annotation5: <Test #x"85 B3026172 B4 B30152 85 B3026166 B30166 84" @ar <R @af f>>
annotation6: <Test #x"B4 85 B3026172 B30152 85 B3026166 B30166 84" <@ar R @af f>>
annotation7:
;Stop reading symbols at @ -- this test has three separate annotations
# Stop reading symbols at @ -- this test has three separate annotations
<Test #x"85 B30161 85 B30162 85 B30163 B584" @a@b@c[]>
bytes2: <Test #x"B20568656c6c6f" #"hello">
bytes2a: <Test @"Internal whitespace is allowed, including commas!" #x"B2, 05, 68, 65, 6c, 6c, 6f" #"hello">