Compare commits
11 Commits
@preserves
...
main
Author | SHA1 | Date |
---|---|---|
Tony Garnock-Jones | 73c6593f84 | |
Tony Garnock-Jones | a9e226f759 | |
Tony Garnock-Jones | 33db0b8718 | |
Tony Garnock-Jones | e923d87fa5 | |
Tony Garnock-Jones | 83697b0e56 | |
Tony Garnock-Jones | 1798e64615 | |
Tony Garnock-Jones | be32f9b7c8 | |
Tony Garnock-Jones | dc1b0ac54d | |
Tony Garnock-Jones | d579a0d607 | |
Tony Garnock-Jones | 7178fb0d9b | |
Tony Garnock-Jones | 4c0bd3b9d7 |
1
Makefile
1
Makefile
|
@ -22,4 +22,3 @@ test-all:
|
|||
(cd implementations/javascript; npm test)
|
||||
(cd implementations/python; make test)
|
||||
(cd implementations/racket/preserves; make testonly)
|
||||
(cd implementations/rust; cargo test)
|
||||
|
|
16
README.md
16
README.md
|
@ -38,14 +38,14 @@ automatic, perfect-fidelity conversion between syntaxes.
|
|||
|
||||
#### Implementations of the data model, plus Preserves textual and binary transfer syntax
|
||||
|
||||
| Language[^pre-alpha-implementations] | Code | Package | Docs |
|
||||
|-----------------------|------------------------------------------------------------------------------|--------------------------------------------------------------------------------|-------------------------------------------|
|
||||
| Nim | [git.syndicate-lang.org](https://git.syndicate-lang.org/ehmry/preserves-nim) | | |
|
||||
| Python | [preserves.dev]({{page.projecttree}}/implementations/python/) | [`pip install preserves`](https://pypi.org/project/preserves/) | [docs](python/latest/) |
|
||||
| Racket | [preserves.dev]({{page.projecttree}}/implementations/racket/preserves/) | [`raco pkg install preserves`](https://pkgs.racket-lang.org/package/preserves) | |
|
||||
| Rust | [preserves.dev]({{page.projecttree}}/implementations/rust/) | [`cargo add preserves`](https://crates.io/crates/preserves) | [docs](https://docs.rs/preserves/latest/) |
|
||||
| Squeak Smalltalk | [SqueakSource](https://squeaksource.com/Preserves.html) | `Installer ss project: 'Preserves';`<br>` install: 'Preserves'` | |
|
||||
| TypeScript/JavaScript | [preserves.dev]({{page.projecttree}}/implementations/javascript/) | [`yarn add @preserves/core`](https://www.npmjs.com/package/@preserves/core) | |
|
||||
| Language[^pre-alpha-implementations] | Code | Package | Docs |
|
||||
|--------------------------------------|------------------------------------------------------------------------------|--------------------------------------------------------------------------------|-------------------------------------------|
|
||||
| Nim | [git.syndicate-lang.org](https://git.syndicate-lang.org/ehmry/preserves-nim) | | |
|
||||
| Python | [preserves.dev]({{page.projecttree}}/implementations/python/) | [`pip install preserves`](https://pypi.org/project/preserves/) | [docs](python/latest/) |
|
||||
| Racket | [preserves.dev]({{page.projecttree}}/implementations/racket/preserves/) | [`raco pkg install preserves`](https://pkgs.racket-lang.org/package/preserves) | |
|
||||
| Rust | [preserves.dev](https://gitlab.com/preserves/preserves-rs/) | [`cargo add preserves`](https://crates.io/crates/preserves) | [docs](https://docs.rs/preserves/latest/) |
|
||||
| Squeak Smalltalk | [SqueakSource](https://squeaksource.com/Preserves.html) | `Installer ss project: 'Preserves';`<br>` install: 'Preserves'` | |
|
||||
| TypeScript/JavaScript | [preserves.dev]({{page.projecttree}}/implementations/javascript/) | [`yarn add @preserves/core`](https://www.npmjs.com/package/@preserves/core) | |
|
||||
|
||||
[^pre-alpha-implementations]: Pre-alpha implementations also exist for
|
||||
[C]({{page.projecttree}}/implementations/c/) and
|
||||
|
|
|
@ -105,7 +105,7 @@ A few more interesting differences:
|
|||
{"dictionaries": "as keys???"}: "well, why not?"}
|
||||
```
|
||||
|
||||
Preserves technically provides a few types of numbers:
|
||||
Preserves technically provides various types of numbers:
|
||||
|
||||
```
|
||||
# Signed Integers
|
||||
|
@ -114,9 +114,6 @@ Preserves technically provides a few types of numbers:
|
|||
5907212309572059846509324862304968273468909473609826340
|
||||
-5907212309572059846509324862304968273468909473609826340
|
||||
|
||||
# Floats (Single-precision IEEE floats) (notice the trailing f)
|
||||
3.1415927f
|
||||
|
||||
# Doubles (Double-precision IEEE floats)
|
||||
3.141592653589793
|
||||
```
|
||||
|
|
|
@ -13,5 +13,5 @@ defaults:
|
|||
layout: page
|
||||
|
||||
title: "Preserves"
|
||||
version_date: "October 2023"
|
||||
version: "0.992.0"
|
||||
version_date: "February 2024"
|
||||
version: "0.994.0"
|
||||
|
|
|
@ -1,5 +1,7 @@
|
|||
[
|
||||
{"version":"0.992.2","title":"0.992.2","aliases":["latest"]},
|
||||
{"version":"0.994.0","title":"0.994.0","aliases":["latest"]},
|
||||
{"version":"0.993.0","title":"0.993.0","aliases":[]},
|
||||
{"version":"0.992.2","title":"0.992.2","aliases":[]},
|
||||
{"version":"0.992.1","title":"0.992.1","aliases":[]},
|
||||
{"version":"0.992.0","title":"0.992.0","aliases":[]},
|
||||
{"version":"0.991.0","title":"0.991.0","aliases":[]},
|
||||
|
|
|
@ -5,9 +5,8 @@ For a value `V`, we write `«V»` for the binary encoding of `V`.
|
|||
«#t» = [0x81]
|
||||
|
||||
«@W V» = [0x85] ++ «W» ++ «V»
|
||||
«#!V» = [0x86] ++ «V»
|
||||
«#:V» = [0x86] ++ «V»
|
||||
|
||||
«V» if V ∈ Float = [0x87, 0x04] ++ binary32(V)
|
||||
«V» if V ∈ Double = [0x87, 0x08] ++ binary64(V)
|
||||
|
||||
«V» if V ∈ SignedInteger = [0xB0] ++ varint(|intbytes(V)|) ++ intbytes(V)
|
||||
|
@ -29,5 +28,4 @@ For a value `V`, we write `«V»` for the binary encoding of `V`.
|
|||
signedBigEndian(n >> 8) ++ [n & 255] otherwise
|
||||
```
|
||||
|
||||
The functions `binary32(F)` and `binary64(D)` yield big-endian 4- and
|
||||
8-byte IEEE 754 binary representations of `F` and `D`, respectively.
|
||||
The function `binary64(D)` yields the big-endian 8-byte IEEE 754 binary representation of `D`.
|
||||
|
|
|
@ -8,10 +8,9 @@ class="postcard-grammar binarysyntax">*V*</span>.
|
|||
|
||||
{:.postcard-grammar.binarysyntax}
|
||||
«`@`*W* *V*» | = | `85` «*W*» «*V*»
|
||||
«`#!`*V*» | = | `86` «*V*»
|
||||
«`#:`*V*» | = | `86` «*V*»
|
||||
|
||||
{:.postcard-grammar.binarysyntax}
|
||||
«*V*» | = | `87``04` **binary32**(*V*) | if *V* ∈ Float
|
||||
«*V*» | = | `87``08` **binary64**(*V*) | if *V* ∈ Double
|
||||
|
||||
{:.postcard-grammar.binarysyntax}
|
||||
|
@ -37,10 +36,9 @@ class="postcard-grammar binarysyntax">*V*</span>.
|
|||
**signedBigEndian**(*n*) | = | <span class="outputish">*n* & 255</span> | if −128 ≤ *n* ≤ 127
|
||||
| | **signedBigEndian**(*n* >> 8) <span class="outputish">*n* & 255</span> | otherwise
|
||||
|
||||
The functions <span class="postcard-grammar binarysyntax">**binary32**(*F*)</span> and <span
|
||||
class="postcard-grammar binarysyntax">**binary64**(*D*)</span> yield big-endian 4- and 8-byte
|
||||
IEEE 754 binary representations of <span class="postcard-grammar binarysyntax">*F*</span> and
|
||||
<span class="postcard-grammar binarysyntax">*D*</span>, respectively.
|
||||
The function <span class="postcard-grammar binarysyntax">**binary64**(*D*)</span> yields the
|
||||
big-endian 8-byte IEEE 754 binary representation of <span class="postcard-grammar
|
||||
binarysyntax">*D*</span>.
|
||||
|
||||
<!--
|
||||
Together, <span class="postcard-grammar binarysyntax">**div**</span> and <span
|
||||
|
|
|
@ -15,7 +15,7 @@ Set := `#{` Expr* Trailer ws `}`
|
|||
|
||||
Trailer := (ws Annotation)*
|
||||
|
||||
Embedded := `#!` SimpleExpr
|
||||
Embedded := `#:` SimpleExpr
|
||||
Annotated := Annotation SimpleExpr
|
||||
Annotation := `@` SimpleExpr | `#` ((space | tab) linecomment) (cr | lf)
|
||||
```
|
||||
|
|
|
@ -18,6 +18,6 @@ The definitions of `Atom`, `ws`, and `linecomment` are as given in the Preserves
|
|||
| *Trailer* | := | (**ws** *Annotation*)<sup>⋆</sup>
|
||||
|
||||
{:.postcard-grammar.textsyntax}
|
||||
| *Embedded* | := | `#!` *SimpleExpr*
|
||||
| *Embedded* | := | `#:` *SimpleExpr*
|
||||
| *Annotated* | := | *Annotation* *SimpleExpr*
|
||||
| *Annotation* | := | `@` *SimpleExpr* | `#` ((**space** | **tab**) *linecomment*) (**cr** | **lf**)
|
||||
|
|
|
@ -9,7 +9,7 @@ Set := `#{` (commas Value)* commas `}`
|
|||
Dictionary := `{` (commas Value ws `:` Value)* commas `}`
|
||||
commas := (ws `,`)* ws
|
||||
|
||||
Embedded := `#!` Value
|
||||
Embedded := `#:` Value
|
||||
Annotated := Annotation Value
|
||||
Annotation := `@` Value | `#` ((space | tab) linecomment) (cr | lf)
|
||||
|
||||
|
@ -21,8 +21,7 @@ ByteString := `#"` binchar* `"`
|
|||
String := `"` («any unicode scalar except `\` or `"`» | escaped | `\"`)* `"`
|
||||
QuotedSymbol := `|` («any unicode scalar except `\` or `|`» | escaped | `\|`)* `|`
|
||||
Symbol := (`A`..`Z` | `a`..`z` | `0`..`9` | sympunct | symuchar)+
|
||||
Number := Float | Double | SignedInteger
|
||||
Float := flt (`f`|`F`) | `#xf"` (ws hex hex)4 ws `"`
|
||||
Number := Double | SignedInteger
|
||||
Double := flt | `#xd"` (ws hex hex)8 ws `"`
|
||||
SignedInteger := int
|
||||
|
||||
|
|
|
@ -11,7 +11,7 @@
|
|||
| **commas** | := | (**ws** `,`)<sup>⋆</sup> **ws** |
|
||||
|
||||
{:.postcard-grammar.textsyntax}
|
||||
| *Embedded* | := | `#!`*Value* |
|
||||
| *Embedded* | := | `#:`*Value* |
|
||||
| *Annotated* | := | *Annotation* *Value* |
|
||||
| *Annotation* | := | `@`*Value* |`#` ((**space** | **tab**) *linecomment*) (**cr** | **lf**) |
|
||||
|
||||
|
@ -22,8 +22,7 @@
|
|||
| *String* | := | `"` (« any unicode scalar value except `\` or `"` » | *escaped* |`\"`)<sup>⋆</sup> `"` |
|
||||
| *QuotedSymbol* | := | `|` (« any unicode scalar value except `\` or `|` » | *escaped* |`\|`)<sup>⋆</sup> `|` |
|
||||
| *Symbol* | := | (`A`..`Z`|`a`..`z`|`0`..`9`| *sympunct* | *symuchar*)<sup>+</sup> |
|
||||
| *Number* | := | *Float* | *Double* | *SignedInteger* |
|
||||
| *Float* | := | *flt* (`f`|`F`) |`#xf"` (**ws** *hex* *hex*)<sup>4</sup> **ws**`"` |
|
||||
| *Number* | := | *Double* | *SignedInteger* |
|
||||
| *Double* | := | *flt* |`#xd"` (**ws** *hex* *hex*)<sup>8</sup> **ws**`"` |
|
||||
| *SignedInteger* | := | *int* |
|
||||
|
||||
|
|
|
@ -1,5 +1,5 @@
|
|||
Python's strings, byte strings, integers, booleans, and double-precision floats stand directly
|
||||
for their Preserves counterparts. Wrapper objects for [Float][preserves.values.Float] and
|
||||
for their Preserves counterparts. Wrapper objects for
|
||||
[Symbol][preserves.values.Symbol] complete the suite of atomic types.
|
||||
|
||||
Python's lists and tuples correspond to Preserves `Sequence`s, and dicts and sets to
|
||||
|
|
|
@ -2,7 +2,6 @@ Here are a few example values, written using the [text
|
|||
syntax](https://preserves.dev/preserves-text.html):
|
||||
|
||||
Boolean : #t #f
|
||||
Float : 1.0f 10.4e3f -100.6f
|
||||
Double : 1.0 10.4e3 -100.6
|
||||
Integer : 1 0 -100
|
||||
String : "Hello, world!\n"
|
||||
|
@ -12,6 +11,6 @@ syntax](https://preserves.dev/preserves-text.html):
|
|||
Sequence : [value1 value2 ...]
|
||||
Set : #{value1 value2 ...}
|
||||
Dictionary : {key1: value1 key2: value2 ...: ...}
|
||||
Embedded : #!value
|
||||
Embedded : #:value
|
||||
|
||||
Commas are optional in sequences, sets, and dictionaries.
|
||||
|
|
|
@ -4,7 +4,6 @@
|
|||
| Embedded
|
||||
|
||||
Atom = Boolean
|
||||
| Float
|
||||
| Double
|
||||
| SignedInteger
|
||||
| String
|
||||
|
|
|
@ -38,7 +38,7 @@ representations of their keys.[^no-need-for-by-value]
|
|||
**Other kinds of `Value`.**
|
||||
There are no special canonicalization restrictions on
|
||||
`SignedInteger`s, `String`s, `ByteString`s, `Symbol`s, `Boolean`s,
|
||||
`Float`s, `Double`s, `Record`s, `Sequence`s, or `Embedded`s. The
|
||||
`Double`s, `Record`s, `Sequence`s, or `Embedded`s. The
|
||||
constraints given for these `Value`s in the [specification][spec]
|
||||
suffice to ensure canonicity.
|
||||
|
||||
|
|
|
@ -23,10 +23,10 @@ Appropriately-labelled `Record`s denote these domain-specific data
|
|||
types.[^why-dictionaries]
|
||||
|
||||
[^why-dictionaries]: Given `Record`'s existence, it may seem odd
|
||||
that `Dictionary`, `Set`, `Float`, etc. are given special
|
||||
that `Dictionary`, `Set`, `Double`, etc. are given special
|
||||
treatment. Preserves aims to offer a useful basic equivalence
|
||||
predicate to programmers, and so if a data type demands a special
|
||||
equivalence predicate, as `Dictionary`, `Set` and `Float` all do,
|
||||
equivalence predicate, as `Dictionary`, `Set` and `Double` all do,
|
||||
then the type should be included in the base language. Otherwise,
|
||||
it can be represented as a `Record` and treated separately.
|
||||
`Boolean`, `String` and `Symbol` are seeming exceptions. The first
|
||||
|
|
|
@ -31,7 +31,6 @@ fi
|
|||
# Ensure that various copies of schema.prs, schema.bin, path.bin,
|
||||
# samples.pr and samples.bin are in fact identical.
|
||||
${COMMAND} path/path.bin implementations/python/preserves/path.prb
|
||||
${COMMAND} path/path.bin implementations/rust/preserves-path/path.bin
|
||||
|
||||
${COMMAND} schema/schema.bin implementations/python/preserves/schema.prb
|
||||
${COMMAND} schema/schema.prs implementations/racket/preserves/preserves-schema/schema.prs
|
||||
|
@ -40,11 +39,4 @@ ${COMMAND} tests/samples.bin implementations/python/tests/samples.bin
|
|||
${COMMAND} tests/samples.pr implementations/python/tests/samples.pr
|
||||
${COMMAND} tests/samples.pr implementations/racket/preserves/preserves/tests/samples.pr
|
||||
|
||||
${COMMAND} _includes/what-is-preserves.md implementations/rust/preserves/doc/what-is-preserves.md
|
||||
${COMMAND} _includes/cheatsheet-binary-plaintext.md implementations/rust/preserves/doc/cheatsheet-binary-plaintext.md
|
||||
${COMMAND} _includes/cheatsheet-text-plaintext.md implementations/rust/preserves/doc/cheatsheet-text-plaintext.md
|
||||
${COMMAND} _includes/value-grammar.md implementations/rust/preserves/doc/value-grammar.md
|
||||
|
||||
${COMMAND} _includes/what-is-preserves-schema.md implementations/rust/preserves-schema/doc/what-is-preserves-schema.md
|
||||
|
||||
[ -z "$failed" ]
|
||||
|
|
|
@ -13,9 +13,9 @@ Here you may find:
|
|||
- [racket](racket/), an implementation for Racket 7.x and newer
|
||||
(though older Rackets may also work with it).
|
||||
|
||||
- [rust](rust/), an implementation for Rust that interoperates with
|
||||
serde.
|
||||
|
||||
Other implementations are also available:
|
||||
|
||||
- [Preserves for Rust](https://gitlab.com/preserves/preserves-rs/), an implementation for Rust
|
||||
that interoperates with serde.
|
||||
|
||||
- [Preserves for Squeak Smalltalk](https://squeaksource.com/Preserves.html)
|
||||
|
|
|
@ -261,7 +261,6 @@ PRESERVES_OUTOFLINE
|
|||
|
||||
typedef enum preserves_type_tag {
|
||||
PRESERVES_BOOLEAN = 0,
|
||||
PRESERVES_FLOAT,
|
||||
PRESERVES_DOUBLE,
|
||||
|
||||
PRESERVES_SIGNED_INTEGER,
|
||||
|
@ -283,7 +282,6 @@ typedef enum preserves_type_tag {
|
|||
PRESERVES_OUTOFLINE(char const *preserves_type_tag_name(preserves_type_tag_t type), {
|
||||
switch (type) {
|
||||
case PRESERVES_BOOLEAN: return "BOOLEAN";
|
||||
case PRESERVES_FLOAT: return "FLOAT";
|
||||
case PRESERVES_DOUBLE: return "DOUBLE";
|
||||
case PRESERVES_SIGNED_INTEGER: return "SIGNED_INTEGER";
|
||||
case PRESERVES_STRING: return "STRING";
|
||||
|
@ -366,7 +364,6 @@ PRESERVES_OUTOFLINE
|
|||
|
||||
/*
|
||||
PRESERVES_BOOLEAN: repr==PRESERVES_REPR_NONE, len=0, data._boolean
|
||||
PRESERVES_FLOAT: repr=PRESERVES_REPR_NONE, len=0, data._float
|
||||
PRESERVES_DOUBLE: repr=PRESERVES_REPR_NONE, len=0, data._double
|
||||
|
||||
PRESERVES_SIGNED_INTEGER:
|
||||
|
@ -418,7 +415,6 @@ typedef struct preserves_index_entry {
|
|||
|
||||
union {
|
||||
bool _boolean;
|
||||
float _float;
|
||||
double _double;
|
||||
int64_t _signed;
|
||||
uint64_t _unsigned;
|
||||
|
@ -818,18 +814,6 @@ PRESERVES_OUTOFLINE
|
|||
uint8_t *bs = _preserves_reader_next_bytes(r, len);
|
||||
if (bs == NULL) return _preserves_reader_finish(r, PRESERVES_END_INCOMPLETE_INPUT);
|
||||
switch (len) {
|
||||
case 4: {
|
||||
uint32_t i;
|
||||
memcpy(&i, bs, 4);
|
||||
i = ntohl(i);
|
||||
float f;
|
||||
memcpy(&f, &i, 4);
|
||||
RETURN_ON_FAIL(_preserves_reader_emit_entry(r, &count, (preserves_index_entry_t) {
|
||||
.type = PRESERVES_FLOAT, .repr = PRESERVES_REPR_NONE, .len = 0, .data = {
|
||||
._float = f
|
||||
}}));
|
||||
break;
|
||||
}
|
||||
case 8: {
|
||||
uint32_t lo, hi;
|
||||
memcpy(&hi, bs, 4);
|
||||
|
@ -995,10 +979,6 @@ PRESERVES_IMPLEMENTATION_CHUNK
|
|||
fprintf(f, i->data._boolean ? " #t" : " #f");
|
||||
break;
|
||||
|
||||
case PRESERVES_FLOAT:
|
||||
fprintf(f, " %f", i->data._float);
|
||||
break;
|
||||
|
||||
case PRESERVES_DOUBLE:
|
||||
fprintf(f, " %f", i->data._double);
|
||||
break;
|
||||
|
|
|
@ -41,15 +41,6 @@ namespace Preserves {
|
|||
decodeEmbedded(decodeEmbedded)
|
||||
{}
|
||||
|
||||
boost::optional<float> next_float() {
|
||||
uint8_t buf[4];
|
||||
if (!next_chunk(buf, sizeof(buf))) return boost::none;
|
||||
uint32_t n = buf[0] << 24 | buf[1] << 16 | buf[2] << 8 | buf[3];
|
||||
float f;
|
||||
memcpy(&f, &n, sizeof(f));
|
||||
return f;
|
||||
}
|
||||
|
||||
boost::optional<double> next_double() {
|
||||
uint8_t buf[8];
|
||||
if (!next_chunk(buf, sizeof(buf))) return boost::none;
|
||||
|
@ -113,7 +104,6 @@ namespace Preserves {
|
|||
return BinaryReader<>(i).next().map(decodeEmbedded).map(Value<T>::from_embedded);
|
||||
case BinaryTag::Ieee754: return varint(i).flat_map([&](size_t len)-> boost::optional<Value<T>> {
|
||||
switch (len) {
|
||||
case 4: return next_float().map(Value<T>::from_float);
|
||||
case 8: return next_double().map(Value<T>::from_double);
|
||||
default: return boost::none;
|
||||
}
|
||||
|
|
|
@ -151,19 +151,6 @@ namespace Preserves {
|
|||
return (*this) << (b ? BinaryTag::True : BinaryTag::False);
|
||||
}
|
||||
|
||||
BinaryWriter& operator<<(float f) {
|
||||
uint32_t n;
|
||||
memcpy(&n, &f, sizeof(f));
|
||||
uint8_t buf[4];
|
||||
buf[0] = (n >> 24) & 0xff;
|
||||
buf[1] = (n >> 16) & 0xff;
|
||||
buf[2] = (n >> 8) & 0xff;
|
||||
buf[3] = (n) & 0xff;
|
||||
(*this) << BinaryTag::Ieee754;
|
||||
put(uint8_t(sizeof(buf)));
|
||||
return write(buf, sizeof(buf));
|
||||
}
|
||||
|
||||
BinaryWriter& operator<<(double d) {
|
||||
uint64_t n;
|
||||
memcpy(&n, &d, sizeof(d));
|
||||
|
|
|
@ -35,13 +35,6 @@ namespace Preserves {
|
|||
BinaryWriter& write(BinaryWriter& w) const override {
|
||||
return w << this->_value();
|
||||
});
|
||||
PRESERVES_ATOMIC_VALUE_CLASS(Float, float, float, ValueKind::Float, as_float,
|
||||
BinaryWriter& write(BinaryWriter& w) const override {
|
||||
return w << this->_value();
|
||||
}
|
||||
boost::optional<double> as_double() const override {
|
||||
return this->value;
|
||||
});
|
||||
PRESERVES_ATOMIC_VALUE_CLASS(Double, double, double, ValueKind::Double, as_double,
|
||||
BinaryWriter& write(BinaryWriter& w) const override {
|
||||
return w << this->_value();
|
||||
|
@ -57,13 +50,6 @@ namespace Preserves {
|
|||
return boost::none;
|
||||
}
|
||||
}
|
||||
boost::optional<float> as_float() const override {
|
||||
if (uint64_t(float(this->value)) == this->value) {
|
||||
return float(this->value);
|
||||
} else {
|
||||
return boost::none;
|
||||
}
|
||||
}
|
||||
boost::optional<double> as_double() const override {
|
||||
if (uint64_t(double(this->value)) == this->value) {
|
||||
return double(this->value);
|
||||
|
@ -82,13 +68,6 @@ namespace Preserves {
|
|||
return boost::none;
|
||||
}
|
||||
}
|
||||
boost::optional<float> as_float() const override {
|
||||
if (int64_t(float(this->value)) == this->value) {
|
||||
return float(this->value);
|
||||
} else {
|
||||
return boost::none;
|
||||
}
|
||||
}
|
||||
boost::optional<double> as_double() const override {
|
||||
if (int64_t(double(this->value)) == this->value) {
|
||||
return double(this->value);
|
||||
|
@ -295,7 +274,6 @@ namespace Preserves {
|
|||
bool is_mutable() const override { return underlying.is_mutable(); }
|
||||
|
||||
boost::optional<bool> as_bool() const override { return underlying.as_bool(); }
|
||||
boost::optional<float> as_float() const override { return underlying.as_float(); }
|
||||
boost::optional<double> as_double() const override { return underlying.as_double(); }
|
||||
boost::optional<uint64_t> as_unsigned() const override { return underlying.as_unsigned(); }
|
||||
boost::optional<int64_t> as_signed() const override { return underlying.as_signed(); }
|
||||
|
@ -355,12 +333,6 @@ namespace Preserves {
|
|||
return Value<T>(new Boolean<T>(b));
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
Value<T> Value<T>::from_float(float f)
|
||||
{
|
||||
return Value<T>(new Float<T>(f));
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
Value<T> Value<T>::from_double(double d)
|
||||
{
|
||||
|
|
|
@ -14,7 +14,6 @@ namespace Preserves {
|
|||
|
||||
enum class ValueKind {
|
||||
Boolean,
|
||||
Float,
|
||||
Double,
|
||||
SignedInteger,
|
||||
String,
|
||||
|
@ -43,7 +42,6 @@ namespace Preserves {
|
|||
std::shared_ptr<ValueImpl<T>> _impl() const { return p; }
|
||||
|
||||
static Value from_bool(bool b);
|
||||
static Value from_float(float f);
|
||||
static Value from_double(double d);
|
||||
static Value from_int(uint64_t i);
|
||||
static Value from_int(int64_t i);
|
||||
|
@ -67,11 +65,9 @@ namespace Preserves {
|
|||
|
||||
static Value from_number(uint64_t i) { return from_int(i); }
|
||||
static Value from_number(int64_t i) { return from_int(i); }
|
||||
static Value from_number(float f) { return from_float(f); }
|
||||
static Value from_number(double d) { return from_double(d); }
|
||||
|
||||
static Value from(bool b) { return from_bool(b); }
|
||||
static Value from(float f) { return from_float(f); }
|
||||
static Value from(double d) { return from_double(d); }
|
||||
static Value from(uint64_t i) { return from_int(i); }
|
||||
static Value from(unsigned i) { return from_int(uint64_t(i)); }
|
||||
|
@ -95,7 +91,6 @@ namespace Preserves {
|
|||
bool is_mutable() const;
|
||||
|
||||
bool is_bool() const { return value_kind() == ValueKind::Boolean; }
|
||||
bool is_float() const { return value_kind() == ValueKind::Float; }
|
||||
bool is_double() const { return value_kind() == ValueKind::Double; }
|
||||
bool is_int() const { return value_kind() == ValueKind::SignedInteger; }
|
||||
bool is_string() const { return value_kind() == ValueKind::String; }
|
||||
|
@ -109,9 +104,6 @@ namespace Preserves {
|
|||
boost::optional<bool> as_bool() const;
|
||||
bool to_bool() const { return as_bool().value(); }
|
||||
|
||||
boost::optional<float> as_float() const;
|
||||
float to_float() const { return as_float().value(); }
|
||||
|
||||
boost::optional<double> as_double() const;
|
||||
double to_double() const { return as_double().value(); }
|
||||
|
||||
|
@ -175,7 +167,6 @@ namespace Preserves {
|
|||
virtual bool is_mutable() const { return false; }
|
||||
|
||||
virtual boost::optional<bool> as_bool() const { return boost::none; }
|
||||
virtual boost::optional<float> as_float() const { return boost::none; }
|
||||
virtual boost::optional<double> as_double() const { return boost::none; }
|
||||
virtual boost::optional<uint64_t> as_unsigned() const { return boost::none; }
|
||||
virtual boost::optional<int64_t> as_signed() const { return boost::none; }
|
||||
|
@ -219,7 +210,6 @@ namespace Preserves {
|
|||
#define PRESERVES_DELEGATE_CAST(t, name) \
|
||||
template <typename T> boost::optional<t> Value<T>::name() const { return p->name(); }
|
||||
PRESERVES_DELEGATE_CAST(bool, as_bool);
|
||||
PRESERVES_DELEGATE_CAST(float, as_float);
|
||||
PRESERVES_DELEGATE_CAST(double, as_double);
|
||||
PRESERVES_DELEGATE_CAST(uint64_t, as_unsigned);
|
||||
PRESERVES_DELEGATE_CAST(int64_t, as_signed);
|
||||
|
@ -265,7 +255,6 @@ namespace Preserves {
|
|||
if (bKind < aKind) return false;
|
||||
switch (aKind) {
|
||||
case ValueKind::Boolean: return a.to_bool() < b.to_bool();
|
||||
case ValueKind::Float: return a.to_float() < b.to_float();
|
||||
case ValueKind::Double: return a.to_double() < b.to_double();
|
||||
case ValueKind::SignedInteger: {
|
||||
if (auto av = a.as_signed()) {
|
||||
|
|
|
@ -41,7 +41,7 @@ let render
|
|||
)
|
||||
m
|
||||
++ " }"
|
||||
, embedded = λ(value : Text) → "#!${value}"
|
||||
, embedded = λ(value : Text) → "#:${value}"
|
||||
}
|
||||
|
||||
let Preserves/boolean = ./boolean.dhall
|
||||
|
@ -94,7 +94,7 @@ let example0 =
|
|||
)}
|
||||
''
|
||||
≡ ''
|
||||
{ a: 1 b: [ 2 3 ] c: { d: 1.0 e: -1.0 } d: #!#t e: <capture <_>> }
|
||||
{ a: 1 b: [ 2 3 ] c: { d: 1.0 e: -1.0 } d: #:#t e: <capture <_>> }
|
||||
''
|
||||
|
||||
in render
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "@preserves/core",
|
||||
"version": "0.992.4",
|
||||
"version": "0.994.0",
|
||||
"description": "Preserves data serialization format",
|
||||
"homepage": "https://gitlab.com/preserves/preserves",
|
||||
"license": "Apache-2.0",
|
||||
|
|
|
@ -2,7 +2,7 @@ import { Annotated } from "./annotated";
|
|||
import { DecodeError, ShortPacket } from "./codec";
|
||||
import { Tag } from "./constants";
|
||||
import { Set, Dictionary } from "./dictionary";
|
||||
import { DoubleFloat, SingleFloat } from "./float";
|
||||
import { DoubleFloat } from "./float";
|
||||
import { Record } from "./record";
|
||||
import { Bytes, BytesLike, underlying, hexDigit } from "./bytes";
|
||||
import { Value } from "./values";
|
||||
|
@ -31,7 +31,6 @@ export interface TypedDecoder<T> {
|
|||
body: (d: TypedDecoder<S>) => R): R;
|
||||
|
||||
nextBoolean(): boolean | undefined;
|
||||
nextFloat(): SingleFloat | undefined;
|
||||
nextDouble(): DoubleFloat | undefined;
|
||||
nextEmbedded(): Embedded<T> | undefined;
|
||||
nextSignedInteger(): number | bigint | undefined;
|
||||
|
@ -241,7 +240,6 @@ export class Decoder<T = never> implements TypedDecoder<T> {
|
|||
case Tag.Embedded: return this.state.wrap<T>(embed(this.embeddedDecode.decode(this.state)));
|
||||
case Tag.Ieee754:
|
||||
switch (this.state.varint()) {
|
||||
case 4: return this.state.wrap<T>(SingleFloat.fromBytes(this.state.nextbytes(4)));
|
||||
case 8: return this.state.wrap<T>(DoubleFloat.fromBytes(this.state.nextbytes(8)));
|
||||
default: throw new DecodeError("Invalid IEEE754 size");
|
||||
}
|
||||
|
@ -308,14 +306,6 @@ export class Decoder<T = never> implements TypedDecoder<T> {
|
|||
});
|
||||
}
|
||||
|
||||
nextFloat(): SingleFloat | undefined {
|
||||
return this.skipAnnotations((reset) => {
|
||||
if (this.state.nextbyte() !== Tag.Ieee754) return reset();
|
||||
if (this.state.nextbyte() !== 4) return reset();
|
||||
return SingleFloat.fromBytes(this.state.nextbytes(4));
|
||||
});
|
||||
}
|
||||
|
||||
nextDouble(): DoubleFloat | undefined {
|
||||
return this.skipAnnotations((reset) => {
|
||||
if (this.state.nextbyte() !== Tag.Ieee754) return reset();
|
||||
|
|
|
@ -26,7 +26,7 @@ export class Embedded<T> {
|
|||
}
|
||||
|
||||
toString(): string {
|
||||
return '#!' + (this.embeddedValue as any).toString();
|
||||
return '#:' + (this.embeddedValue as any).toString();
|
||||
}
|
||||
|
||||
__as_preserve__<R>(): T extends R ? Value<R> : never {
|
||||
|
|
|
@ -5,7 +5,8 @@ import type { Encoder, Preservable } from "./encoder";
|
|||
import type { Writer, PreserveWritable } from "./writer";
|
||||
import { Bytes, dataview } from "./bytes";
|
||||
|
||||
export type FloatType = 'Single' | 'Double';
|
||||
// v Previously included 'Single'; may again in future. Also, 'Half', 'Quad'?
|
||||
export type FloatType = 'Double';
|
||||
export const FloatType = Symbol.for('FloatType');
|
||||
|
||||
export abstract class Float {
|
||||
|
@ -37,7 +38,6 @@ export abstract class Float {
|
|||
abstract get [FloatType](): FloatType;
|
||||
|
||||
static isFloat = (x: any): x is Float => x?.[FloatType] !== void 0;
|
||||
static isSingle = (x: any): x is SingleFloat => x?.[FloatType] === 'Single';
|
||||
static isDouble = (x: any): x is DoubleFloat => x?.[FloatType] === 'Double';
|
||||
}
|
||||
|
||||
|
@ -58,76 +58,39 @@ export function floatlikeString(f: number): string {
|
|||
return s + '.0';
|
||||
}
|
||||
|
||||
export class SingleFloat extends Float implements Preservable<any>, PreserveWritable<any> {
|
||||
__as_preserve__<T = GenericEmbedded>(): Value<T> {
|
||||
return this;
|
||||
}
|
||||
|
||||
static fromBytes(bs: Bytes | DataView): SingleFloat {
|
||||
const view = dataview(bs);
|
||||
const vf = view.getInt32(0, false);
|
||||
if ((vf & 0x7f800000) === 0x7f800000) {
|
||||
// NaN or inf. Preserve quiet/signalling bit by manually expanding to double-precision.
|
||||
const sign = vf >> 31;
|
||||
const payload = vf & 0x007fffff;
|
||||
const dbs = new Bytes(8);
|
||||
const dview = dataview(dbs);
|
||||
dview.setInt16(0, (sign << 15) | 0x7ff0 | (payload >> 19), false);
|
||||
dview.setInt32(2, (payload & 0x7ffff) << 13, false);
|
||||
return new SingleFloat(dview.getFloat64(0, false));
|
||||
} else {
|
||||
return new SingleFloat(dataview(bs).getFloat32(0, false));
|
||||
}
|
||||
}
|
||||
|
||||
static __from_preserve__<T>(v: Value<T>): undefined | SingleFloat {
|
||||
return Float.isSingle(v) ? v : void 0;
|
||||
}
|
||||
|
||||
__w(v: DataView, offset: number) {
|
||||
if (Number.isNaN(this.value)) {
|
||||
const dbs = new Bytes(8);
|
||||
const dview = dataview(dbs);
|
||||
dview.setFloat64(0, this.value, false);
|
||||
const sign = dview.getInt8(0) >> 7;
|
||||
const payload = (dview.getInt32(1, false) >> 5) & 0x007fffff;
|
||||
const vf = (sign << 31) | 0x7f800000 | payload;
|
||||
v.setInt32(offset, vf, false);
|
||||
} else {
|
||||
v.setFloat32(offset, this.value, false);
|
||||
}
|
||||
}
|
||||
|
||||
__preserve_on__(encoder: Encoder<any>) {
|
||||
encoder.state.emitbyte(Tag.Ieee754);
|
||||
encoder.state.emitbyte(4);
|
||||
encoder.state.makeroom(4);
|
||||
this.__w(encoder.state.view, encoder.state.index);
|
||||
encoder.state.index += 4;
|
||||
}
|
||||
|
||||
toBytes(): Bytes {
|
||||
const bs = new Bytes(4);
|
||||
this.__w(bs.dataview(), 0);
|
||||
return bs;
|
||||
}
|
||||
|
||||
toString(): string {
|
||||
if (Number.isFinite(this.value)) {
|
||||
return floatlikeString(this.value) + 'f';
|
||||
} else {
|
||||
return '#xf"' + this.toBytes().toHex() + '"';
|
||||
}
|
||||
}
|
||||
|
||||
get [FloatType](): 'Single' {
|
||||
return 'Single';
|
||||
}
|
||||
}
|
||||
|
||||
export function Single(value: number | Float): SingleFloat {
|
||||
return new SingleFloat(value);
|
||||
}
|
||||
// -- These snippets are useful to keep in mind for promoting 4-byte, single-precision floats
|
||||
// -- to 8-byte, double-precision floats *while preserving NaN bit-patterns*:
|
||||
//
|
||||
// static fromBytes(bs: Bytes | DataView): SingleFloat {
|
||||
// const view = dataview(bs);
|
||||
// const vf = view.getInt32(0, false);
|
||||
// if ((vf & 0x7f800000) === 0x7f800000) {
|
||||
// // NaN or inf. Preserve quiet/signalling bit by manually expanding to double-precision.
|
||||
// const sign = vf >> 31;
|
||||
// const payload = vf & 0x007fffff;
|
||||
// const dbs = new Bytes(8);
|
||||
// const dview = dataview(dbs);
|
||||
// dview.setInt16(0, (sign << 15) | 0x7ff0 | (payload >> 19), false);
|
||||
// dview.setInt32(2, (payload & 0x7ffff) << 13, false);
|
||||
// return new SingleFloat(dview.getFloat64(0, false));
|
||||
// } else {
|
||||
// return new SingleFloat(dataview(bs).getFloat32(0, false));
|
||||
// }
|
||||
// }
|
||||
//
|
||||
// __w(v: DataView, offset: number) {
|
||||
// if (Number.isNaN(this.value)) {
|
||||
// const dbs = new Bytes(8);
|
||||
// const dview = dataview(dbs);
|
||||
// dview.setFloat64(0, this.value, false);
|
||||
// const sign = dview.getInt8(0) >> 7;
|
||||
// const payload = (dview.getInt32(1, false) >> 5) & 0x007fffff;
|
||||
// const vf = (sign << 31) | 0x7f800000 | payload;
|
||||
// v.setInt32(offset, vf, false);
|
||||
// } else {
|
||||
// v.setFloat32(offset, this.value, false);
|
||||
// }
|
||||
// }
|
||||
|
||||
export class DoubleFloat extends Float implements Preservable<any>, PreserveWritable<any> {
|
||||
__as_preserve__<T = GenericEmbedded>(): Value<T> {
|
||||
|
|
|
@ -3,12 +3,11 @@ import { Bytes } from "./bytes";
|
|||
import { Value } from "./values";
|
||||
import { Set, Dictionary } from "./dictionary";
|
||||
import { annotate, Annotated } from "./annotated";
|
||||
import { Double, Float, Single } from "./float";
|
||||
import { Double, Float } from "./float";
|
||||
import { Embedded } from "./embedded";
|
||||
|
||||
export enum ValueClass {
|
||||
Boolean,
|
||||
Float,
|
||||
Double,
|
||||
SignedInteger,
|
||||
String,
|
||||
|
@ -26,7 +25,6 @@ export type Fold<T, R = Value<T>> = (v: Value<T>) => R;
|
|||
|
||||
export interface FoldMethods<T, R> {
|
||||
boolean(b: boolean): R;
|
||||
single(f: number): R;
|
||||
double(f: number): R;
|
||||
integer(i: number | bigint): R;
|
||||
string(s: string): R;
|
||||
|
@ -45,7 +43,6 @@ export interface FoldMethods<T, R> {
|
|||
|
||||
export class VoidFold<T> implements FoldMethods<T, void> {
|
||||
boolean(b: boolean): void {}
|
||||
single(f: number): void {}
|
||||
double(f: number): void {}
|
||||
integer(i: number | bigint): void {}
|
||||
string(s: string): void {}
|
||||
|
@ -73,9 +70,6 @@ export abstract class ValueFold<T, R = T> implements FoldMethods<T, Value<R>> {
|
|||
boolean(b: boolean): Value<R> {
|
||||
return b;
|
||||
}
|
||||
single(f: number): Value<R> {
|
||||
return Single(f);
|
||||
}
|
||||
double(f: number): Value<R> {
|
||||
return Double(f);
|
||||
}
|
||||
|
@ -134,7 +128,7 @@ export function valueClass<T>(v: Value<T>): ValueClass {
|
|||
return ValueClass.Boolean;
|
||||
case 'number':
|
||||
if (!Number.isInteger(v)) {
|
||||
throw new Error("Non-integer number in Preserves valueClass; missing SingleFloat/DoubleFloat wrapper?");
|
||||
throw new Error("Non-integer number in Preserves valueClass; missing Float wrapper?");
|
||||
} else {
|
||||
return ValueClass.SignedInteger;
|
||||
}
|
||||
|
@ -157,8 +151,6 @@ export function valueClass<T>(v: Value<T>): ValueClass {
|
|||
return ValueClass.Annotated;
|
||||
} else if (Bytes.isBytes(v)) {
|
||||
return ValueClass.ByteString;
|
||||
} else if (Float.isSingle(v)) {
|
||||
return ValueClass.Float;
|
||||
} else if (Float.isDouble(v)) {
|
||||
return ValueClass.Double;
|
||||
} else {
|
||||
|
@ -202,8 +194,6 @@ export function fold<T, R>(v: Value<T>, o: FoldMethods<T, R>): R {
|
|||
return o.annotated(v, walk);
|
||||
} else if (Bytes.isBytes(v)) {
|
||||
return o.bytes(v);
|
||||
} else if (Float.isSingle(v)) {
|
||||
return o.single(v.value);
|
||||
} else if (Float.isDouble(v)) {
|
||||
return o.double(v.value);
|
||||
} else {
|
||||
|
|
|
@ -9,7 +9,7 @@ export function fromJS<T = GenericEmbedded>(x: any): Value<T> {
|
|||
case 'number':
|
||||
if (!Number.isInteger(x)) {
|
||||
// We require that clients be explicit about integer vs. non-integer types.
|
||||
throw new TypeError("Refusing to autoconvert non-integer number to Single or Double");
|
||||
throw new TypeError("Refusing to autoconvert non-integer number to Double");
|
||||
}
|
||||
// FALL THROUGH
|
||||
case 'bigint':
|
||||
|
|
|
@ -32,7 +32,6 @@ export function merge<T>(
|
|||
}
|
||||
return fold<T, Value<T>>(a, {
|
||||
boolean: die,
|
||||
single(_f: number) { return is(a, b) ? a : die(); },
|
||||
double(_f: number) { return is(a, b) ? a : die(); },
|
||||
integer: die,
|
||||
string: die,
|
||||
|
|
|
@ -19,9 +19,7 @@ export function typeCode<V>(v: Value<V>): number {
|
|||
case 'symbol':
|
||||
return 6;
|
||||
case 'object':
|
||||
if (Float.isFloat(v)) {
|
||||
return Float.isSingle(v) ? 1 : 2;
|
||||
}
|
||||
if (Float.isFloat(v)) return 2; // 1 was for single-precision floats
|
||||
if (Bytes.isBytes(v)) return 5;
|
||||
if (Array.isArray(v)) {
|
||||
return ('label' in v) ? 7 : 8;
|
||||
|
@ -55,7 +53,7 @@ export function compare<V>(
|
|||
const vb = b as any;
|
||||
return va < vb ? -1 : va > vb ? 1 : 0;
|
||||
}
|
||||
case 1:
|
||||
// case 1: // was single-precision
|
||||
case 2: {
|
||||
const va = (a as Float).value;
|
||||
const vb = (b as Float).value;
|
||||
|
|
|
@ -8,7 +8,7 @@ import { Bytes, unhexDigit } from './bytes';
|
|||
import { Decoder, DecoderState, neverEmbeddedTypeDecode } from './decoder';
|
||||
import { Record } from './record';
|
||||
import { Annotated, newPosition, Position, updatePosition } from './annotated';
|
||||
import { Double, DoubleFloat, FloatType, Single, SingleFloat } from './float';
|
||||
import { Double, DoubleFloat, FloatType } from './float';
|
||||
import { stringify } from './text';
|
||||
import { embed, GenericEmbedded, EmbeddedTypeDecode } from './embedded';
|
||||
|
||||
|
@ -24,12 +24,10 @@ export interface ReaderOptions<T> extends ReaderStateOptions {
|
|||
const MAX_SAFE_INTEGERn = BigInt(Number.MAX_SAFE_INTEGER);
|
||||
const MIN_SAFE_INTEGERn = BigInt(Number.MIN_SAFE_INTEGER);
|
||||
|
||||
export const NUMBER_RE: RegExp = /^([-+]?\d+)(((\.\d+([eE][-+]?\d+)?)|([eE][-+]?\d+))([fF]?))?$/;
|
||||
export const NUMBER_RE: RegExp = /^([-+]?\d+)((\.\d+([eE][-+]?\d+)?)|([eE][-+]?\d+))?$/;
|
||||
// Groups:
|
||||
// 1 - integer part and sign
|
||||
// 2 - decimal part, exponent and Float marker
|
||||
// 3 - decimal part and exponent
|
||||
// 7 - Float marker
|
||||
// 2 - decimal part and exponent
|
||||
|
||||
export class ReaderState {
|
||||
buffer: string;
|
||||
|
@ -131,20 +129,14 @@ export class ReaderState {
|
|||
}
|
||||
}
|
||||
|
||||
readHexFloat(precision: FloatType): SingleFloat | DoubleFloat {
|
||||
readHexFloat(): DoubleFloat {
|
||||
const pos = this.copyPos();
|
||||
if (this.nextchar() !== '"') {
|
||||
this.error("Missing open-double-quote in hex-encoded floating-point number", pos);
|
||||
}
|
||||
const bs = this.readHexBinary();
|
||||
switch (precision) {
|
||||
case 'Single':
|
||||
if (bs.length !== 4) this.error("Incorrect number of bytes in hex-encoded Float", pos);
|
||||
return SingleFloat.fromBytes(bs);
|
||||
case 'Double':
|
||||
if (bs.length !== 8) this.error("Incorrect number of bytes in hex-encoded Double", pos);
|
||||
return DoubleFloat.fromBytes(bs);
|
||||
}
|
||||
if (bs.length !== 8) this.error("Incorrect number of bytes in hex-encoded Double", pos);
|
||||
return DoubleFloat.fromBytes(bs);
|
||||
}
|
||||
|
||||
readBase64Binary(): Bytes {
|
||||
|
@ -180,10 +172,8 @@ export class ReaderState {
|
|||
} else {
|
||||
return Number(v);
|
||||
}
|
||||
} else if (m[7] === '') {
|
||||
return Double(parseFloat(m[1] + m[3]));
|
||||
} else {
|
||||
return Single(parseFloat(m[1] + m[3]));
|
||||
return Double(parseFloat(acc));
|
||||
}
|
||||
} else {
|
||||
return Symbol.for(acc);
|
||||
|
@ -360,12 +350,11 @@ export class Reader<T> {
|
|||
case '"': return this.state.readLiteralBinary();
|
||||
case 'x': switch (this.state.nextchar()) {
|
||||
case '"': return this.state.readHexBinary();
|
||||
case 'f': return this.state.readHexFloat('Single');
|
||||
case 'd': return this.state.readHexFloat('Double');
|
||||
case 'd': return this.state.readHexFloat();
|
||||
default: this.state.error('Invalid #x syntax', startPos);
|
||||
}
|
||||
case '[': return this.state.readBase64Binary();
|
||||
case '!': return embed(this.embeddedType.fromValue(
|
||||
case ':': return embed(this.embeddedType.fromValue(
|
||||
new Reader<GenericEmbedded>(this.state, genericEmbeddedTypeDecode).next(),
|
||||
this.state.options));
|
||||
default:
|
||||
|
|
|
@ -1,7 +1,7 @@
|
|||
// Preserves Values.
|
||||
|
||||
import type { Bytes } from './bytes';
|
||||
import type { DoubleFloat, SingleFloat } from './float';
|
||||
import type { DoubleFloat } from './float';
|
||||
import type { Annotated } from './annotated';
|
||||
import type { Set, Dictionary } from './dictionary';
|
||||
import type { Embedded, GenericEmbedded } from './embedded';
|
||||
|
@ -13,7 +13,6 @@ export type Value<T = GenericEmbedded> =
|
|||
| Annotated<T>;
|
||||
export type Atom =
|
||||
| boolean
|
||||
| SingleFloat
|
||||
| DoubleFloat
|
||||
| number | bigint
|
||||
| string
|
||||
|
|
|
@ -297,7 +297,7 @@ export class Writer<T> {
|
|||
}
|
||||
else {
|
||||
((v: Embedded<T>) => {
|
||||
this.state.pieces.push('#!');
|
||||
this.state.pieces.push('#:');
|
||||
if ('write' in this.embeddedWrite) {
|
||||
this.embeddedWrite.write(this.state, v.embeddedValue);
|
||||
} else {
|
||||
|
|
|
@ -1,12 +1,6 @@
|
|||
import { Single, Double, fromJS, Dictionary, IDENTITY_FOLD, fold, mapEmbeddeds, Value, embed, preserves } from '../src/index';
|
||||
import { Double, fromJS, Dictionary, IDENTITY_FOLD, fold, mapEmbeddeds, Value, embed, preserves } from '../src/index';
|
||||
import './test-utils';
|
||||
|
||||
describe('Single', () => {
|
||||
it('should print reasonably', () => {
|
||||
expect(Single(123.45).toString()).toEqual("123.45f");
|
||||
});
|
||||
});
|
||||
|
||||
describe('Double', () => {
|
||||
it('should print reasonably', () => {
|
||||
expect(Double(123.45).toString()).toEqual("123.45");
|
||||
|
@ -21,7 +15,7 @@ describe('fold', () => {
|
|||
new Dictionary([[[3, 4], fromJS([5, 6])],
|
||||
['a', 1],
|
||||
['b', true]]),
|
||||
Single(3.4),
|
||||
Double(3.4),
|
||||
t,
|
||||
]);
|
||||
}
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "@preserves/schema-cli",
|
||||
"version": "0.992.4",
|
||||
"version": "0.994.0",
|
||||
"description": "Command-line tools for Preserves Schema",
|
||||
"homepage": "https://gitlab.com/preserves/preserves",
|
||||
"license": "Apache-2.0",
|
||||
|
@ -26,8 +26,8 @@
|
|||
"@types/minimatch": "^3.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@preserves/core": "^0.992.4",
|
||||
"@preserves/schema": "^0.992.4",
|
||||
"@preserves/core": "^0.994.0",
|
||||
"@preserves/schema": "^0.994.0",
|
||||
"chalk": "^4.1",
|
||||
"chokidar": "^3.5",
|
||||
"commander": "^7.2",
|
||||
|
|
|
@ -24,10 +24,9 @@ export function run(options: CommandLineArguments): void {
|
|||
|
||||
if (failures.length === 0) {
|
||||
if (options.bundle) {
|
||||
fs.writeSync(1, underlying(canonicalEncode(M.fromBundle({
|
||||
modules: new KeyedDictionary<M.ModulePath, M.Schema, M.InputEmbedded>(
|
||||
inputFiles.map(i => [i.modulePath, i.schema])),
|
||||
}))));
|
||||
fs.writeSync(1, underlying(canonicalEncode(M.fromBundle(M.Bundle(
|
||||
new KeyedDictionary<M.ModulePath, M.Schema, M.InputEmbedded>(
|
||||
inputFiles.map(i => [i.modulePath, i.schema])))))));
|
||||
} else {
|
||||
fs.writeSync(1, underlying(canonicalEncode(M.fromSchema(inputFiles[0].schema))));
|
||||
}
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "@preserves/schema",
|
||||
"version": "0.992.4",
|
||||
"version": "0.994.0",
|
||||
"description": "Schema support for Preserves data serialization format",
|
||||
"homepage": "https://gitlab.com/preserves/preserves",
|
||||
"license": "Apache-2.0",
|
||||
|
@ -20,12 +20,12 @@
|
|||
"compile:watch": "yarn compile -w",
|
||||
"rollup": "rollup -c",
|
||||
"rollup:watch": "yarn rollup -w",
|
||||
"copy-schema": "mkdir -p ./dist && cp -a ../../../../schema/*.prs ./dist",
|
||||
"copy-schema": "mkdir -p ./dist && cp -a ../../../../schema/schema.prs ../../../../schema/host.prs ./dist",
|
||||
"test": "jest",
|
||||
"test:watch": "jest --watch",
|
||||
"veryclean": "yarn run clean && rm -rf node_modules"
|
||||
},
|
||||
"dependencies": {
|
||||
"@preserves/core": "^0.992.4"
|
||||
"@preserves/core": "^0.994.0"
|
||||
}
|
||||
}
|
||||
|
|
|
@ -154,7 +154,6 @@ export function converterForSimple(
|
|||
let valexp: Item = `${src}`;
|
||||
switch (p.atomKind._variant) {
|
||||
case 'Boolean': test = `typeof ${src} === 'boolean'`; break;
|
||||
case 'Float': test = `_.Float.isSingle(${src})`; valexp = `${src}.value`; break;
|
||||
case 'Double': test =`_.Float.isDouble(${src})`; valexp = `${src}.value`; break;
|
||||
case 'SignedInteger': test = `typeof ${src} === 'number'`; break;
|
||||
case 'String': test = `typeof ${src} === 'string'`; break;
|
||||
|
|
|
@ -37,7 +37,6 @@ export function simpleType(resolver: RefResolver, p: M.SimplePattern): FieldType
|
|||
case 'atom':
|
||||
switch (p.atomKind._variant) {
|
||||
case 'Boolean': return Type.ref(`boolean`, null);
|
||||
case 'Float': return Type.ref(`number`, null);
|
||||
case 'Double': return Type.ref(`number`, null);
|
||||
case 'SignedInteger': return Type.ref(`number`, null);
|
||||
case 'String': return Type.ref(`string`, null);
|
||||
|
|
|
@ -40,7 +40,6 @@ function unconverterFor(ctx: FunctionContext, p: M.Pattern, src: string): Item {
|
|||
return `${src}`;
|
||||
case 'atom':
|
||||
switch (p.atomKind._variant) {
|
||||
case 'Float': return `_.Single(${src})`;
|
||||
case 'Double': return `_.Double(${src})`;
|
||||
default: return `${src}`;
|
||||
}
|
||||
|
|
|
@ -5,7 +5,6 @@ import * as M from '../meta';
|
|||
export function sourceCodeFor(v: Value<M.InputEmbedded>): Item {
|
||||
return fold(v, {
|
||||
boolean(b: boolean): Item { return b.toString(); },
|
||||
single(f: number): Item { return f.toString(); },
|
||||
double(f: number): Item { return f.toString(); },
|
||||
integer(i: number): Item { return i.toString(); },
|
||||
string(s: string): Item { return JSON.stringify(s); },
|
||||
|
|
File diff suppressed because one or more lines are too long
|
@ -1,4 +1,5 @@
|
|||
import { EncodableDictionary, KeyedDictionary, Dictionary, Value, is, Record, Float, Bytes, isEmbedded, isSequence, Set, Atom, Embedded, merge as plainMerge, Preservable, PreserveWritable, _iterMap, stringify } from '@preserves/core';
|
||||
import { EncodableDictionary, KeyedDictionary, Dictionary, Value, is, Record, Float, Bytes, isEmbedded, isSequence, Set, Atom, Embedded, merge as plainMerge, Preservable, PreserveWritable, _iterMap, stringify, fromJS } from '@preserves/core';
|
||||
import { SchemaDefinition } from './reflection';
|
||||
import * as M from './meta';
|
||||
import * as H from './host';
|
||||
|
||||
|
@ -15,12 +16,9 @@ export type BindingName = string;
|
|||
export type Bindings<V> = { [key: BindingName]: Parsed<V> };
|
||||
export type TopBindings<V> = Bindings<V> & Top<V>;
|
||||
|
||||
export type SingleConstructor<V> = (input: any) => Parsed<V>;
|
||||
export type SingleConstructor<V> = ((input: any) => Parsed<V>) & { schema(): SchemaDefinition };
|
||||
export type MultipleConstructors<V> = { [key: string]: SingleConstructor<V> };
|
||||
export type DefinitionConstructors<V> =
|
||||
| { single: SingleConstructor<V> }
|
||||
| { multiple: MultipleConstructors<V> }
|
||||
;
|
||||
export type DefinitionConstructors<V> = SingleConstructor<V> | MultipleConstructors<V>;
|
||||
|
||||
export namespace Bindings {
|
||||
export function empty<V>(): Bindings<V> {
|
||||
|
@ -111,14 +109,18 @@ export class SchemaInterpreter<V> {
|
|||
}
|
||||
}
|
||||
|
||||
_lookup<R>(modulePath: M.ModulePath, name: symbol, f: (d: M.Definition<V>) => R): R {
|
||||
_lookup<R>(
|
||||
modulePath: M.ModulePath,
|
||||
name: symbol,
|
||||
f: (d: M.Definition<V>, schema: M.Schema<V>) => R,
|
||||
): R {
|
||||
const { resolved, schema } = this._findModule(modulePath);
|
||||
return this._withModule(resolved, () => {
|
||||
const definition = schema.definitions.get(name);
|
||||
if (definition === void 0) {
|
||||
throw new Error(`No such preserves-schema definition: ${[... modulePath, name].map(s => s.description!).join('.')}`);
|
||||
}
|
||||
return f(definition);
|
||||
return f(definition, schema);
|
||||
});
|
||||
}
|
||||
|
||||
|
@ -134,7 +136,20 @@ export class SchemaInterpreter<V> {
|
|||
modulePath: M.ModulePath,
|
||||
name: symbol,
|
||||
): DefinitionConstructors<V> {
|
||||
return this._lookup(modulePath, name, (definition): DefinitionConstructors<V> => {
|
||||
return this._lookup(modulePath, name, (definition, schema): DefinitionConstructors<V> => {
|
||||
function attachSchema(
|
||||
f: (input: any) => Parsed<V>,
|
||||
variant?: symbol,
|
||||
): SingleConstructor<V> {
|
||||
const g = f as SingleConstructor<V>;
|
||||
g.schema = () => ({
|
||||
schema: fromJS(schema),
|
||||
imports: {}, // TODO
|
||||
definitionName: name,
|
||||
variant,
|
||||
});
|
||||
return g;
|
||||
}
|
||||
const ty = H.definitionType(definition);
|
||||
if (ty._variant === 'union') {
|
||||
const multiple: MultipleConstructors<V> = {};
|
||||
|
@ -142,27 +157,38 @@ export class SchemaInterpreter<V> {
|
|||
const _variant = v.label.description!;
|
||||
switch (v.type._variant) {
|
||||
case 'Field':
|
||||
multiple[_variant] = (value: any) =>
|
||||
this.makeTop(modulePath, name, { _variant, value });
|
||||
multiple[_variant] = attachSchema(
|
||||
(value: any) => this.makeTop(modulePath, name, { _variant, value }),
|
||||
v.label);
|
||||
break;
|
||||
case 'Record':
|
||||
multiple[_variant] = (fields: object) =>
|
||||
this.makeTop(modulePath, name, { _variant, ... fields });
|
||||
multiple[_variant] = attachSchema(
|
||||
(fields: object) => this.makeTop(modulePath, name, { _variant, ... fields }),
|
||||
v.label);
|
||||
break;
|
||||
}
|
||||
});
|
||||
return { multiple };
|
||||
return multiple;
|
||||
} else {
|
||||
const flatName = M.formatModulePath([... modulePath, name]);
|
||||
switch (ty.value._variant) {
|
||||
case 'Field': {
|
||||
const tmp = { [flatName]: (value: any) => value };
|
||||
return { single: tmp[flatName] };
|
||||
return attachSchema(tmp[flatName]);
|
||||
}
|
||||
case 'Record': {
|
||||
const tmp = { [flatName]: (fields: Bindings<V>) =>
|
||||
this.makeTop(modulePath, name, fields) };
|
||||
return { single: tmp[flatName] };
|
||||
const rec = ty.value.value;
|
||||
if (rec.fields.length > 1) {
|
||||
const tmp = { [flatName]: (fields: Bindings<V>) =>
|
||||
this.makeTop(modulePath, name, fields) };
|
||||
return attachSchema(tmp[flatName]);
|
||||
} else {
|
||||
const tmp = { [flatName]: (field: Parsed<V>) =>
|
||||
this.makeTop(modulePath, name, {
|
||||
[M.jsId(rec.fields[0].name.description!)]: field,
|
||||
}) };
|
||||
return attachSchema(tmp[flatName]);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -259,7 +285,6 @@ export class SchemaInterpreter<V> {
|
|||
case 'any': return input;
|
||||
case 'atom': switch (p.atomKind._variant) {
|
||||
case 'Boolean': return inputIf(typeof input === 'boolean');
|
||||
case 'Float': return inputIf(Float.isFloat(input));
|
||||
case 'Double': return inputIf(Float.isDouble(input));
|
||||
case 'SignedInteger': return inputIf(typeof input === 'number' || typeof input === 'bigint');
|
||||
case 'String': return inputIf(typeof input === 'string');
|
||||
|
|
|
@ -225,7 +225,6 @@ function parsePattern(name: symbol, body0: Array<Input>): Pattern {
|
|||
switch (str) {
|
||||
case 'any': return ks(M.SimplePattern.any());
|
||||
case 'bool': return ks(M.SimplePattern.atom(M.AtomKind.Boolean()));
|
||||
case 'float': return ks(M.SimplePattern.atom(M.AtomKind.Float()));
|
||||
case 'double': return ks(M.SimplePattern.atom(M.AtomKind.Double()));
|
||||
case 'int': return ks(M.SimplePattern.atom(M.AtomKind.SignedInteger()));
|
||||
case 'string': return ks(M.SimplePattern.atom(M.AtomKind.String()));
|
||||
|
|
|
@ -18,7 +18,7 @@ describe('reader schema', () => {
|
|||
expect(s.embeddedType._variant).toBe('false');
|
||||
});
|
||||
it('understands patterns under embed', () => {
|
||||
const s = readSchema('version 1 . X = #!0 .');
|
||||
const s = readSchema('version 1 . X = #:0 .');
|
||||
const def: Meta.Definition = s.definitions.get(Symbol.for('X'))!;
|
||||
if (def._variant !== 'Pattern') fail('bad definition 1');
|
||||
if (def.value._variant !== 'SimplePattern') fail ('bad definition 2');
|
||||
|
|
|
@ -7,7 +7,6 @@ The main package re-exports a subset of the exports of its constituent modules:
|
|||
- From [preserves.values][]:
|
||||
- [Annotated][preserves.values.Annotated]
|
||||
- [Embedded][preserves.values.Embedded]
|
||||
- [Float][preserves.values.Float]
|
||||
- [ImmutableDict][preserves.values.ImmutableDict]
|
||||
- [Record][preserves.values.Record]
|
||||
- [Symbol][preserves.values.Symbol]
|
||||
|
@ -56,7 +55,7 @@ Finally, it provides a few utility aliases for common tasks:
|
|||
|
||||
'''
|
||||
|
||||
from .values import Float, Symbol, Record, ImmutableDict, Embedded, preserve
|
||||
from .values import Symbol, Record, ImmutableDict, Embedded, preserve
|
||||
|
||||
from .values import Annotated, is_annotated, strip_annotations, annotate
|
||||
|
||||
|
|
|
@ -206,7 +206,6 @@ class Decoder(BinaryCodec):
|
|||
return self.wrap(Embedded(self.decode_embedded(self.next())))
|
||||
if tag == 0x87:
|
||||
count = self.nextbyte()
|
||||
if count == 4: return self.wrap(Float.from_bytes(self.nextbytes(4)))
|
||||
if count == 8: return self.wrap(struct.unpack('>d', self.nextbytes(8))[0])
|
||||
raise DecodeError('Invalid IEEE754 size')
|
||||
if tag == 0xb0: return self.wrap(self.nextint(self.varint()))
|
||||
|
|
|
@ -33,12 +33,12 @@ import numbers
|
|||
from enum import Enum
|
||||
from functools import cmp_to_key
|
||||
|
||||
from .values import preserve, Float, Embedded, Record, Symbol, cmp_floats, _unwrap
|
||||
from .values import preserve, Embedded, Record, Symbol, cmp_floats, _unwrap
|
||||
from .compat import basestring_
|
||||
|
||||
class TypeNumber(Enum):
|
||||
BOOL = 0
|
||||
FLOAT = 1
|
||||
# FLOAT = 1 # single-precision
|
||||
DOUBLE = 2
|
||||
SIGNED_INTEGER = 3
|
||||
STRING = 4
|
||||
|
@ -57,7 +57,6 @@ def type_number(v):
|
|||
raise ValueError('type_number expects Preserves value; use preserve()')
|
||||
|
||||
if isinstance(v, bool): return TypeNumber.BOOL
|
||||
if isinstance(v, Float): return TypeNumber.FLOAT
|
||||
if isinstance(v, float): return TypeNumber.DOUBLE
|
||||
if isinstance(v, numbers.Number): return TypeNumber.SIGNED_INTEGER
|
||||
if isinstance(v, basestring_): return TypeNumber.STRING
|
||||
|
|
|
@ -9,7 +9,7 @@ def map_embeddeds(f, v):
|
|||
|
||||
```python
|
||||
>>> map_embeddeds(lambda w: Embedded(f'w={w}'), ['a', Embedded(123), {'z': 6.0}])
|
||||
('a', #!'w=123', {'z': 6.0})
|
||||
('a', #:'w=123', {'z': 6.0})
|
||||
|
||||
```
|
||||
"""
|
||||
|
|
|
@ -1,5 +1,5 @@
|
|||
´³schema·³version°³definitions·³Axis´³orµµ±values´³rec´³lit³values„´³tupleµ„„„„µ±descendants´³rec´³lit³descendants„´³tupleµ„„„„µ±at´³rec´³lit³at„´³tupleµ´³named³key³any„„„„„µ±label´³rec´³lit³label„´³tupleµ„„„„µ±keys´³rec´³lit³keys„´³tupleµ„„„„µ±length´³rec´³lit³length„´³tupleµ„„„„µ±annotations´³rec´³lit³annotations„´³tupleµ„„„„µ±embedded´³rec´³lit³embedded„´³tupleµ„„„„µ±parse´³rec´³lit³parse„´³tupleµ´³named³module´³seqof´³atom³Symbol„„„´³named³name´³atom³Symbol„„„„„„µ±unparse´³rec´³lit³unparse„´³tupleµ´³named³module´³seqof´³atom³Symbol„„„´³named³name´³atom³Symbol„„„„„„„„³Step´³orµµ±Axis´³refµ„³Axis„„µ±Filter´³refµ„³Filter„„µ±Function´³refµ„³Function„„„„³Filter´³orµµ±nop´³rec´³lit³nop„´³tupleµ„„„„µ±compare´³rec´³lit³compare„´³tupleµ´³named³op´³refµ„³
|
||||
Comparison„„´³named³literal³any„„„„„µ±regex´³rec´³lit³regex„´³tupleµ´³named³regex´³atom³String„„„„„„µ±test´³rec´³lit³test„´³tupleµ´³named³pred´³refµ„³ Predicate„„„„„„µ±real´³rec´³lit³real„´³tupleµ„„„„µ±int´³rec´³lit³int„´³tupleµ„„„„µ±kind´³rec´³lit³kind„´³tupleµ´³named³kind´³refµ„³ ValueKind„„„„„„„„³Function´³rec´³lit³count„´³tupleµ´³named³selector´³refµ„³Selector„„„„„³Selector´³seqof´³refµ„³Step„„³ Predicate´³orµµ±Selector´³refµ„³Selector„„µ±not´³rec´³lit³not„´³tupleµ´³named³pred´³refµ„³ Predicate„„„„„„µ±or´³rec´³lit³or„´³tupleµ´³named³preds´³seqof´³refµ„³ Predicate„„„„„„„µ±and´³rec´³lit³and„´³tupleµ´³named³preds´³seqof´³refµ„³ Predicate„„„„„„„„„³ ValueKind´³orµµ±Boolean´³lit³Boolean„„µ±Float´³lit³Float„„µ±Double´³lit³Double„„µ±
SignedInteger´³lit³
SignedInteger„„µ±String´³lit³String„„µ±
|
||||
Comparison„„´³named³literal³any„„„„„µ±regex´³rec´³lit³regex„´³tupleµ´³named³regex´³atom³String„„„„„„µ±test´³rec´³lit³test„´³tupleµ´³named³pred´³refµ„³ Predicate„„„„„„µ±real´³rec´³lit³real„´³tupleµ„„„„µ±int´³rec´³lit³int„´³tupleµ„„„„µ±kind´³rec´³lit³kind„´³tupleµ´³named³kind´³refµ„³ ValueKind„„„„„„„„³Function´³rec´³lit³count„´³tupleµ´³named³selector´³refµ„³Selector„„„„„³Selector´³seqof´³refµ„³Step„„³ Predicate´³orµµ±Selector´³refµ„³Selector„„µ±not´³rec´³lit³not„´³tupleµ´³named³pred´³refµ„³ Predicate„„„„„„µ±or´³rec´³lit³or„´³tupleµ´³named³preds´³seqof´³refµ„³ Predicate„„„„„„„µ±and´³rec´³lit³and„´³tupleµ´³named³preds´³seqof´³refµ„³ Predicate„„„„„„„„„³ ValueKind´³orµµ±Boolean´³lit³Boolean„„µ±Double´³lit³Double„„µ±
SignedInteger´³lit³
SignedInteger„„µ±String´³lit³String„„µ±
|
||||
ByteString´³lit³
|
||||
ByteString„„µ±Symbol´³lit³Symbol„„µ±Record´³lit³Record„„µ±Sequence´³lit³Sequence„„µ±Set´³lit³Set„„µ±
|
||||
Dictionary´³lit³
|
||||
|
|
|
@ -170,7 +170,6 @@ FILTER_RE2 = Symbol('=r')
|
|||
FILTER_LABEL = Symbol('^')
|
||||
|
||||
FILTER_BOOL = Symbol('bool')
|
||||
FILTER_FLOAT = Symbol('float')
|
||||
FILTER_DOUBLE = Symbol('double')
|
||||
FILTER_INT = Symbol('int')
|
||||
FILTER_STRING = Symbol('string')
|
||||
|
@ -226,7 +225,6 @@ def parse_step(tokens):
|
|||
if t == TRANSFORM_REAL: return syntax.Step.Filter(syntax.Filter.real)
|
||||
if t == TRANSFORM_INT: return syntax.Step.Filter(syntax.Filter.int)
|
||||
if t == FILTER_BOOL: return kind_filter(syntax.ValueKind.Boolean())
|
||||
if t == FILTER_FLOAT: return kind_filter(syntax.ValueKind.Float())
|
||||
if t == FILTER_DOUBLE: return kind_filter(syntax.ValueKind.Double())
|
||||
if t == FILTER_INT: return kind_filter(syntax.ValueKind.SignedInteger())
|
||||
if t == FILTER_STRING: return kind_filter(syntax.ValueKind.String())
|
||||
|
@ -448,8 +446,6 @@ def exec(self, v):
|
|||
@extend(syntax.Filter.real)
|
||||
def exec(self, v):
|
||||
v = preserve(_unwrap(v))
|
||||
if isinstance(v, Float):
|
||||
return (v.value,)
|
||||
if type(v) == float:
|
||||
return (v,)
|
||||
if type(v) == int:
|
||||
|
@ -459,8 +455,6 @@ def exec(self, v):
|
|||
@extend(syntax.Filter.int)
|
||||
def exec(self, v):
|
||||
v = preserve(_unwrap(v))
|
||||
if isinstance(v, Float):
|
||||
return (int(v.value()),)
|
||||
if type(v) == float:
|
||||
return (int(v),)
|
||||
if type(v) == int:
|
||||
|
@ -476,10 +470,6 @@ def exec(self, v):
|
|||
def exec(self, v):
|
||||
return (v,) if type(v) == bool else ()
|
||||
|
||||
@extend(syntax.ValueKind.Float)
|
||||
def exec(self, v):
|
||||
return (v,) if isinstance(v, Float) else ()
|
||||
|
||||
@extend(syntax.ValueKind.Double)
|
||||
def exec(self, v):
|
||||
return (v,) if type(v) == float else ()
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
´³schema·³version°³definitions·³Ref´³rec´³lit³ref„´³tupleµ´³named³module´³refµ„³
|
||||
ModulePath„„´³named³name´³atom³Symbol„„„„„³Bundle´³rec´³lit³bundle„´³tupleµ´³named³modules´³refµ„³Modules„„„„„³Schema´³rec´³lit³schema„´³tupleµ´³dict·³version´³named³version´³refµ„³Version„„³definitions´³named³definitions´³refµ„³Definitions„„³embeddedType´³named³embeddedType´³refµ„³EmbeddedTypeName„„„„„„„³Binding´³rec´³lit³named„´³tupleµ´³named³name´³atom³Symbol„„´³named³pattern´³refµ„³
SimplePattern„„„„„³Modules´³dictof´³refµ„³
|
||||
ModulePath„´³refµ„³Schema„„³Pattern´³orµµ±
SimplePattern´³refµ„³
SimplePattern„„µ±CompoundPattern´³refµ„³CompoundPattern„„„„³Version´³lit°„³AtomKind´³orµµ±Boolean´³lit³Boolean„„µ±Float´³lit³Float„„µ±Double´³lit³Double„„µ±
SignedInteger´³lit³
SignedInteger„„µ±String´³lit³String„„µ±
|
||||
ModulePath„´³refµ„³Schema„„³Pattern´³orµµ±
SimplePattern´³refµ„³
SimplePattern„„µ±CompoundPattern´³refµ„³CompoundPattern„„„„³Version´³lit°„³AtomKind´³orµµ±Boolean´³lit³Boolean„„µ±Double´³lit³Double„„µ±
SignedInteger´³lit³
SignedInteger„„µ±String´³lit³String„„µ±
|
||||
ByteString´³lit³
|
||||
ByteString„„µ±Symbol´³lit³Symbol„„„„³
|
||||
Definition´³orµµ±or´³rec´³lit³or„´³tupleµ´³tuplePrefixµ´³named³pattern0´³refµ„³NamedAlternative„„´³named³pattern1´³refµ„³NamedAlternative„„„´³named³patternN´³seqof´³refµ„³NamedAlternative„„„„„„„„µ±and´³rec´³lit³and„´³tupleµ´³tuplePrefixµ´³named³pattern0´³refµ„³NamedPattern„„´³named³pattern1´³refµ„³NamedPattern„„„´³named³patternN´³seqof´³refµ„³NamedPattern„„„„„„„„µ±Pattern´³refµ„³Pattern„„„„³
|
||||
|
|
|
@ -279,7 +279,6 @@ DICT = Symbol('dict')
|
|||
DICTOF = Symbol('dictof')
|
||||
DOUBLE = Symbol('Double')
|
||||
EMBEDDED = Symbol('embedded')
|
||||
FLOAT = Symbol('Float')
|
||||
LIT = Symbol('lit')
|
||||
NAMED = Symbol('named')
|
||||
OR = Symbol('or')
|
||||
|
@ -469,7 +468,6 @@ class SchemaObject:
|
|||
if p.key == ATOM:
|
||||
k = p[0]
|
||||
if k == BOOLEAN and isinstance(v, bool): return v
|
||||
if k == FLOAT and isinstance(v, Float): return v
|
||||
if k == DOUBLE and isinstance(v, float): return v
|
||||
if k == SIGNED_INTEGER and isinstance(v, int): return v
|
||||
if k == STRING and isinstance(v, str): return v
|
||||
|
|
|
@ -28,7 +28,7 @@ from .binary import Decoder
|
|||
class TextCodec(object):
|
||||
pass
|
||||
|
||||
NUMBER_RE = re.compile(r'^([-+]?\d+)(((\.\d+([eE][-+]?\d+)?)|([eE][-+]?\d+))([fF]?))?$')
|
||||
NUMBER_RE = re.compile(r'^([-+]?\d+)((\.\d+([eE][-+]?\d+)?)|([eE][-+]?\d+))?$')
|
||||
|
||||
class Parser(TextCodec):
|
||||
"""Parser for the human-readable Preserves text syntax.
|
||||
|
@ -251,15 +251,13 @@ class Parser(TextCodec):
|
|||
if c == '=': continue
|
||||
acc.append(c)
|
||||
|
||||
def read_hex_float(self, bytecount):
|
||||
def read_hex_float(self):
|
||||
if self.nextchar() != '"':
|
||||
raise DecodeError('Missing open-double-quote in hex-encoded floating-point number')
|
||||
bs = self.read_hex_binary()
|
||||
if len(bs) != bytecount:
|
||||
if len(bs) != 8:
|
||||
raise DecodeError('Incorrect number of bytes in hex-encoded floating-point number')
|
||||
if bytecount == 4: return Float.from_bytes(bs)
|
||||
if bytecount == 8: return struct.unpack('>d', bs)[0]
|
||||
raise DecodeError('Unsupported byte count in hex-encoded floating-point number')
|
||||
return struct.unpack('>d', bs)[0]
|
||||
|
||||
def upto(self, delimiter, skip_commas):
|
||||
vs = []
|
||||
|
@ -308,10 +306,8 @@ class Parser(TextCodec):
|
|||
if m:
|
||||
if m[2] is None:
|
||||
return int(m[1])
|
||||
elif m[7] == '':
|
||||
return float(m[1] + m[3])
|
||||
else:
|
||||
return Float(float(m[1] + m[3]))
|
||||
return float(acc)
|
||||
else:
|
||||
return Symbol(acc)
|
||||
|
||||
|
@ -357,11 +353,10 @@ class Parser(TextCodec):
|
|||
if c == 'x':
|
||||
c = self.nextchar()
|
||||
if c == '"': return self.wrap(self.read_hex_binary())
|
||||
if c == 'f': return self.wrap(self.read_hex_float(4))
|
||||
if c == 'd': return self.wrap(self.read_hex_float(8))
|
||||
if c == 'd': return self.wrap(self.read_hex_float())
|
||||
raise DecodeError('Invalid #x syntax')
|
||||
if c == '[': return self.wrap(self.read_base64_binary())
|
||||
if c == '!':
|
||||
if c == ':':
|
||||
if self.parse_embedded is None:
|
||||
raise DecodeError('No parse_embedded function supplied')
|
||||
return self.wrap(Embedded(self.parse_embedded(self.next())))
|
||||
|
|
|
@ -39,135 +39,6 @@ def cmp_floats(a, b):
|
|||
if b & 0x8000000000000000: b = b ^ 0x7fffffffffffffff
|
||||
return a - b
|
||||
|
||||
class Float(object):
|
||||
"""Wrapper for treating a Python double-precision floating-point value as a
|
||||
single-precision (32-bit) float, from Preserves' perspective. (Python lacks native
|
||||
single-precision floating point support.)
|
||||
|
||||
```python
|
||||
>>> Float(3.45)
|
||||
Float(3.45)
|
||||
>>> import preserves
|
||||
>>> preserves.stringify(Float(3.45))
|
||||
'3.45f'
|
||||
>>> preserves.stringify(3.45)
|
||||
'3.45'
|
||||
>>> preserves.parse('3.45f')
|
||||
Float(3.45)
|
||||
>>> preserves.parse('3.45')
|
||||
3.45
|
||||
>>> preserves.encode(Float(3.45))
|
||||
b'\\x87\\x04@\\\\\\xcc\\xcd'
|
||||
>>> preserves.encode(3.45)
|
||||
b'\\x87\\x08@\\x0b\\x99\\x99\\x99\\x99\\x99\\x9a'
|
||||
|
||||
```
|
||||
|
||||
Attributes:
|
||||
value (float): the double-precision representation of intended single-precision value
|
||||
"""
|
||||
def __init__(self, value):
|
||||
self.value = value
|
||||
|
||||
def __eq__(self, other):
|
||||
other = _unwrap(other)
|
||||
if other.__class__ is self.__class__:
|
||||
return cmp_floats(self.value, other.value) == 0
|
||||
|
||||
def __lt__(self, other):
|
||||
other = _unwrap(other)
|
||||
if other.__class__ is self.__class__:
|
||||
return cmp_floats(self.value, other.value) < 0
|
||||
|
||||
def __ne__(self, other):
|
||||
return not self.__eq__(other)
|
||||
|
||||
def __hash__(self):
|
||||
return hash(self.value)
|
||||
|
||||
def __repr__(self):
|
||||
return 'Float(' + repr(self.value) + ')'
|
||||
|
||||
def to_bytes(self):
|
||||
"""Converts this 32-bit single-precision floating point value to its binary32 format,
|
||||
taking care to preserve the quiet/signalling bit-pattern of NaN values, unlike its
|
||||
`struct.pack('>f', ...)` equivalent.
|
||||
|
||||
```python
|
||||
>>> Float.from_bytes(b'\\x7f\\x80\\x00{')
|
||||
Float(nan)
|
||||
>>> Float.from_bytes(b'\\x7f\\x80\\x00{').to_bytes()
|
||||
b'\\x7f\\x80\\x00{'
|
||||
|
||||
>>> struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]
|
||||
nan
|
||||
>>> Float(struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]).to_bytes()
|
||||
b'\\x7f\\xc0\\x00{'
|
||||
>>> struct.pack('>f', struct.unpack('>f', b'\\x7f\\x80\\x00{')[0])
|
||||
b'\\x7f\\xc0\\x00{'
|
||||
|
||||
```
|
||||
|
||||
(Note well the difference between `7f80007b` and `7fc0007b`!)
|
||||
|
||||
"""
|
||||
|
||||
if math.isnan(self.value) or math.isinf(self.value):
|
||||
dbs = struct.pack('>d', self.value)
|
||||
vd = struct.unpack('>Q', dbs)[0]
|
||||
sign = vd >> 63
|
||||
payload = (vd >> 29) & 0x007fffff
|
||||
vf = (sign << 31) | 0x7f800000 | payload
|
||||
return struct.pack('>I', vf)
|
||||
else:
|
||||
return struct.pack('>f', self.value)
|
||||
|
||||
def __preserve_write_binary__(self, encoder):
|
||||
encoder.buffer.append(0x87)
|
||||
encoder.buffer.append(4)
|
||||
encoder.buffer.extend(self.to_bytes())
|
||||
|
||||
def __preserve_write_text__(self, formatter):
|
||||
if math.isnan(self.value) or math.isinf(self.value):
|
||||
formatter.chunks.append('#xf"' + self.to_bytes().hex() + '"')
|
||||
else:
|
||||
formatter.chunks.append(repr(self.value) + 'f')
|
||||
|
||||
@staticmethod
|
||||
def from_bytes(bs):
|
||||
"""Converts a 4-byte-long byte string to a 32-bit single-precision floating point value
|
||||
wrapped in a [Float][preserves.values.Float] instance. Takes care to preserve the
|
||||
quiet/signalling bit-pattern of NaN values, unlike its `struct.unpack('>f', ...)`
|
||||
equivalent.
|
||||
|
||||
```python
|
||||
>>> Float.from_bytes(b'\\x7f\\x80\\x00{')
|
||||
Float(nan)
|
||||
>>> Float.from_bytes(b'\\x7f\\x80\\x00{').to_bytes()
|
||||
b'\\x7f\\x80\\x00{'
|
||||
|
||||
>>> struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]
|
||||
nan
|
||||
>>> Float(struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]).to_bytes()
|
||||
b'\\x7f\\xc0\\x00{'
|
||||
>>> struct.pack('>f', struct.unpack('>f', b'\\x7f\\x80\\x00{')[0])
|
||||
b'\\x7f\\xc0\\x00{'
|
||||
|
||||
```
|
||||
|
||||
(Note well the difference between `7f80007b` and `7fc0007b`!)
|
||||
|
||||
"""
|
||||
vf = struct.unpack('>I', bs)[0]
|
||||
if (vf & 0x7f800000) == 0x7f800000:
|
||||
# NaN or inf. Preserve quiet/signalling bit by manually expanding to double-precision.
|
||||
sign = vf >> 31
|
||||
payload = vf & 0x007fffff
|
||||
dbs = struct.pack('>Q', (sign << 63) | 0x7ff0000000000000 | (payload << 29))
|
||||
return Float(struct.unpack('>d', dbs)[0])
|
||||
else:
|
||||
return Float(struct.unpack('>f', bs)[0])
|
||||
|
||||
# FIXME: This regular expression is conservatively correct, but Anglo-chauvinistic.
|
||||
RAW_SYMBOL_RE = re.compile(r'^[-a-zA-Z0-9~!$%^&*?_=+/.]+$')
|
||||
|
||||
|
@ -704,7 +575,7 @@ class Embedded:
|
|||
>>> import io
|
||||
>>> e = Embedded(io.StringIO('some text'))
|
||||
>>> e # doctest: +ELLIPSIS
|
||||
#!<_io.StringIO object at ...>
|
||||
#:<_io.StringIO object at ...>
|
||||
>>> e.embeddedValue # doctest: +ELLIPSIS
|
||||
<_io.StringIO object at ...>
|
||||
|
||||
|
@ -717,7 +588,7 @@ class Embedded:
|
|||
...
|
||||
TypeError: Cannot preserves-format: None
|
||||
>>> print(preserves.stringify(Embedded(None), format_embedded=lambda x: 'abcdef'))
|
||||
#!"abcdef"
|
||||
#:"abcdef"
|
||||
|
||||
```
|
||||
|
||||
|
@ -739,12 +610,12 @@ class Embedded:
|
|||
return hash(self.embeddedValue)
|
||||
|
||||
def __repr__(self):
|
||||
return '#!%r' % (self.embeddedValue,)
|
||||
return '#:%r' % (self.embeddedValue,)
|
||||
|
||||
def __preserve_write_binary__(self, encoder):
|
||||
encoder.buffer.append(0x86)
|
||||
encoder.append(encoder.encode_embedded(self.embeddedValue))
|
||||
|
||||
def __preserve_write_text__(self, formatter):
|
||||
formatter.chunks.append('#!')
|
||||
formatter.chunks.append('#:')
|
||||
formatter.append(formatter.format_embedded(self.embeddedValue))
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
[project]
|
||||
name = "preserves"
|
||||
version = "0.992.2"
|
||||
version = "0.994.0"
|
||||
description = "Data serialization format"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.6, <4"
|
||||
|
|
|
@ -0,0 +1,73 @@
|
|||
The code below deals with expansion of single-precision IEEE 754 floating point values to
|
||||
double-precision (and vice-versa) without losing detail of NaN bit-patterns.
|
||||
|
||||
```python
|
||||
def to_bytes(self):
|
||||
"""Converts this 32-bit single-precision floating point value to its binary32 format,
|
||||
taking care to preserve the quiet/signalling bit-pattern of NaN values, unlike its
|
||||
`struct.pack('>f', ...)` equivalent.
|
||||
|
||||
```python
|
||||
>>> Float.from_bytes(b'\\x7f\\x80\\x00{')
|
||||
Float(nan)
|
||||
>>> Float.from_bytes(b'\\x7f\\x80\\x00{').to_bytes()
|
||||
b'\\x7f\\x80\\x00{'
|
||||
|
||||
>>> struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]
|
||||
nan
|
||||
>>> Float(struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]).to_bytes()
|
||||
b'\\x7f\\xc0\\x00{'
|
||||
>>> struct.pack('>f', struct.unpack('>f', b'\\x7f\\x80\\x00{')[0])
|
||||
b'\\x7f\\xc0\\x00{'
|
||||
|
||||
```
|
||||
|
||||
(Note well the difference between `7f80007b` and `7fc0007b`!)
|
||||
|
||||
"""
|
||||
|
||||
if math.isnan(self.value) or math.isinf(self.value):
|
||||
dbs = struct.pack('>d', self.value)
|
||||
vd = struct.unpack('>Q', dbs)[0]
|
||||
sign = vd >> 63
|
||||
payload = (vd >> 29) & 0x007fffff
|
||||
vf = (sign << 31) | 0x7f800000 | payload
|
||||
return struct.pack('>I', vf)
|
||||
else:
|
||||
return struct.pack('>f', self.value)
|
||||
|
||||
@staticmethod
|
||||
def from_bytes(bs):
|
||||
"""Converts a 4-byte-long byte string to a 32-bit single-precision floating point value
|
||||
wrapped in a [Float][preserves.values.Float] instance. Takes care to preserve the
|
||||
quiet/signalling bit-pattern of NaN values, unlike its `struct.unpack('>f', ...)`
|
||||
equivalent.
|
||||
|
||||
```python
|
||||
>>> Float.from_bytes(b'\\x7f\\x80\\x00{')
|
||||
Float(nan)
|
||||
>>> Float.from_bytes(b'\\x7f\\x80\\x00{').to_bytes()
|
||||
b'\\x7f\\x80\\x00{'
|
||||
|
||||
>>> struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]
|
||||
nan
|
||||
>>> Float(struct.unpack('>f', b'\\x7f\\x80\\x00{')[0]).to_bytes()
|
||||
b'\\x7f\\xc0\\x00{'
|
||||
>>> struct.pack('>f', struct.unpack('>f', b'\\x7f\\x80\\x00{')[0])
|
||||
b'\\x7f\\xc0\\x00{'
|
||||
|
||||
```
|
||||
|
||||
(Note well the difference between `7f80007b` and `7fc0007b`!)
|
||||
|
||||
"""
|
||||
vf = struct.unpack('>I', bs)[0]
|
||||
if (vf & 0x7f800000) == 0x7f800000:
|
||||
# NaN or inf. Preserve quiet/signalling bit by manually expanding to double-precision.
|
||||
sign = vf >> 31
|
||||
payload = vf & 0x007fffff
|
||||
dbs = struct.pack('>Q', (sign << 63) | 0x7ff0000000000000 | (payload << 29))
|
||||
return Float(struct.unpack('>d', dbs)[0])
|
||||
else:
|
||||
return Float(struct.unpack('>f', bs)[0])
|
||||
```
|
Binary file not shown.
|
@ -119,32 +119,6 @@
|
|||
double15: @"-sNaN" <Test #x"8708fff8000000000111" #xd"fff8000000000111">
|
||||
double16: @"+sNaN" <Test #x"87087ff8000000000001" #xd"7ff8000000000001">
|
||||
double17: @"+sNaN" <Test #x"87087ff8000000000111" #xd"7ff8000000000111">
|
||||
float0: <Test #x"870400000000" 0.0f>
|
||||
float+0: <Test #x"870400000000" +0.0f>
|
||||
float-0: <Test #x"870480000000" -0.0f>
|
||||
float1: <Test #x"87043f800000" 1.0f>
|
||||
float1u: <Test #x"87043f800000" 1.0F>
|
||||
float1a: <Test #x"87043f800000" 1e0f>
|
||||
float1b: <Test #x"87043f800000" 1.0e0f>
|
||||
float1c: <Test #x"87043f800000" 1e-0f>
|
||||
float1d: <Test #x"87043f800000" 1.0e-0f>
|
||||
float1e: <Test #x"87043f800000" 1e+0f>
|
||||
float1f: <Test #x"87043f800000" 1.0e+0f>
|
||||
float2: <Test #x"870412345678" #xf"12 34 56 78">
|
||||
float3: @"Fewer than 8 digits" <ParseError "#xf\"123456\"">
|
||||
float4: @"More than 8 digits" <ParseError "#xf\"123456789a\"">
|
||||
float5: @"Invalid chars" <ParseError "#xf\"12zz5678\"">
|
||||
float6: @"Positive infinity" <Test #x"87047f800000" #xf"7f800000">
|
||||
float7: @"Negative infinity" <Test #x"8704ff800000" #xf"ff800000">
|
||||
float8: @"+sNaN" <Test #x"87047f800001" #xf"7f800001">
|
||||
float9: @"+sNaN" <Test #x"87047f800111" #xf"7f800111">
|
||||
float10: @"-sNaN" <Test #x"8704ff800001" #xf"ff800001">
|
||||
float11: @"-sNaN" <Test #x"8704ff800111" #xf"ff800111">
|
||||
float12: @"Bad spacing" <ParseError "#xf\"12345 678\"">
|
||||
float13: @"+qNaN" <Test #x"87047fc00001" #xf"7fc00001">
|
||||
float14: @"+qNaN" <Test #x"87047fc00111" #xf"7fc00111">
|
||||
float15: @"-qNaN" <Test #x"8704ffc00001" #xf"ffc00001">
|
||||
float16: @"-qNaN" <Test #x"8704ffc00111" #xf"ffc00111">
|
||||
int-98765432109876543210987654321098765432109: <Test #x"b012feddc125aed4226c770369269596ce3f0ad3" -98765432109876543210987654321098765432109>
|
||||
int-12345678123456781234567812345678: <Test #x"b00eff642cf6684f11d1dad08c4a10b2" -12345678123456781234567812345678>
|
||||
int-1234567812345678123456781234567: <Test #x"b00df06ae570d4b4fb62ae746dce79" -1234567812345678123456781234567>
|
||||
|
@ -193,9 +167,9 @@
|
|||
list11: <Test #x"b5b0010184" [01]>
|
||||
list12: <Test #x"b5b0010c84" [12]>
|
||||
noinput0: @"No input at all" <DecodeEOF #x"">
|
||||
embed0: <Test #x"86b000" #!0>
|
||||
embed1: <Test #x"8686b000" #!#!0>
|
||||
embed2: <Test #x"b586b00086b10568656c6c6f84" [#!0 #!"hello"]>
|
||||
embed0: <Test #x"86b000" #:0>
|
||||
embed1: <Test #x"8686b000" #:#:0>
|
||||
embed2: <Test #x"b586b00086b10568656c6c6f84" [#:0 #:"hello"]>
|
||||
record1: <Test #x"b4 b30763617074757265 b4 b30764697363617264 84 84" <capture <discard>>>
|
||||
record2: <Test #x"b4 b3076f627365727665 b4 b305737065616b b4 b30764697363617264 84 b4 b30763617074757265 b4 b30764697363617264 84 84 84 84" <observe <speak <discard> <capture <discard>>>>>
|
||||
record2a: @"Commas not allowed in records" <ParseError "<observe <speak <discard>, <capture <discard>>>>">
|
||||
|
|
|
@ -135,7 +135,6 @@ class BinaryCodecTests(PreservesTestCase):
|
|||
self._roundtrip(131072, _buf(0xb0, 0x03, 0x02, 0x00, 0x00))
|
||||
|
||||
def test_floats(self):
|
||||
self._roundtrip(Float(1.0), _buf(0x87, 0x04, 0x3f, 0x80, 0, 0))
|
||||
self._roundtrip(1.0, _buf(0x87, 0x08, 0x3f, 0xf0, 0, 0, 0, 0, 0, 0))
|
||||
self._roundtrip(-1.202e300, _buf(0x87, 0x08, 0xfe, 0x3c, 0xb7, 0xb7, 0x59, 0xbf, 0x04, 0x26))
|
||||
|
||||
|
|
|
@ -14,7 +14,6 @@
|
|||
(define :encode-embedded values)
|
||||
(define (AtomKind? p)
|
||||
(or (AtomKind-Boolean? p)
|
||||
(AtomKind-Float? p)
|
||||
(AtomKind-Double? p)
|
||||
(AtomKind-SignedInteger? p)
|
||||
(AtomKind-String? p)
|
||||
|
@ -29,15 +28,6 @@
|
|||
((define/generic *->preserve ->preserve)
|
||||
(define (->preserve preservable)
|
||||
(match preservable ((AtomKind-Boolean) 'Boolean)))))
|
||||
(struct
|
||||
AtomKind-Float
|
||||
()
|
||||
#:transparent
|
||||
#:methods
|
||||
gen:preservable
|
||||
((define/generic *->preserve ->preserve)
|
||||
(define (->preserve preservable)
|
||||
(match preservable ((AtomKind-Float) 'Float)))))
|
||||
(struct
|
||||
AtomKind-Double
|
||||
()
|
||||
|
@ -87,7 +77,6 @@
|
|||
(match
|
||||
input
|
||||
((and dest 'Boolean) (AtomKind-Boolean))
|
||||
((and dest 'Float) (AtomKind-Float))
|
||||
((and dest 'Double) (AtomKind-Double))
|
||||
((and dest 'SignedInteger) (AtomKind-SignedInteger))
|
||||
((and dest 'String) (AtomKind-String))
|
||||
|
|
|
@ -27,8 +27,7 @@
|
|||
(define (->preserve preservable)
|
||||
(for/hash [((k v) (in-hash preservable))]
|
||||
(values (*->preserve k) (*->preserve v))))])
|
||||
#:defaults ([float? (define (->preserve preservable) preservable)]
|
||||
[embedded? (define (->preserve preservable) preservable)]
|
||||
#:defaults ([embedded? (define (->preserve preservable) preservable)]
|
||||
[record?
|
||||
(define/generic *->preserve ->preserve)
|
||||
(define (->preserve preservable)
|
||||
|
|
|
@ -26,7 +26,6 @@
|
|||
(maybe-dest dest-pat-stx
|
||||
`(? ,(match atom-kind
|
||||
[(AtomKind-Boolean) 'boolean?]
|
||||
[(AtomKind-Float) 'float?]
|
||||
[(AtomKind-Double) 'flonum?]
|
||||
[(AtomKind-SignedInteger) 'exact-integer?]
|
||||
[(AtomKind-String) 'string?]
|
||||
|
|
|
@ -131,7 +131,6 @@
|
|||
(match (peel-annotations item)
|
||||
['any (ks (SimplePattern-any))]
|
||||
['bool (ks (SimplePattern-atom (AtomKind-Boolean)))]
|
||||
['float (ks (SimplePattern-atom (AtomKind-Float)))]
|
||||
['double (ks (SimplePattern-atom (AtomKind-Double)))]
|
||||
['int (ks (SimplePattern-atom (AtomKind-SignedInteger)))]
|
||||
['string (ks (SimplePattern-atom (AtomKind-String)))]
|
||||
|
|
|
@ -38,10 +38,10 @@ SimplePattern =
|
|||
# any
|
||||
/ =any
|
||||
|
||||
# special builtins: bool, float, double, int, string, bytes, symbol
|
||||
# special builtins: bool, double, int, string, bytes, symbol
|
||||
/ <atom @atomKind AtomKind>
|
||||
|
||||
# matches an embedded value in the input: #!p
|
||||
# matches an embedded value in the input: #:p
|
||||
/ <embedded @interface SimplePattern>
|
||||
|
||||
# =symbol, <<lit> any>, or plain non-symbol atom
|
||||
|
@ -79,7 +79,7 @@ CompoundPattern =
|
|||
|
||||
DictionaryEntries = { any: NamedSimplePattern ...:... }.
|
||||
|
||||
AtomKind = =Boolean / =Float / =Double / =SignedInteger / =String / =ByteString / =Symbol .
|
||||
AtomKind = =Boolean / =Double / =SignedInteger / =String / =ByteString / =Symbol .
|
||||
|
||||
NamedAlternative = [@variantLabel string @pattern Pattern].
|
||||
|
||||
|
|
|
@ -14,7 +14,6 @@
|
|||
(match (unwrap pattern)
|
||||
[(Binding n p) (pattern->unparser p (escape n))]
|
||||
[(SimplePattern-any) `(*->preserve ,src-stx)]
|
||||
[(SimplePattern-atom (AtomKind-Float)) `(->float (*->preserve ,src-stx))]
|
||||
[(SimplePattern-atom (AtomKind-Double)) `(exact->inexact (*->preserve ,src-stx))]
|
||||
[(SimplePattern-atom _) `(*->preserve ,src-stx)]
|
||||
[(SimplePattern-embedded _interface) `(embedded ,src-stx)]
|
||||
|
|
|
@ -9,7 +9,6 @@
|
|||
bytes->double
|
||||
double->bytes)
|
||||
|
||||
(require "float.rkt")
|
||||
(require (only-in racket/math nan? infinite?))
|
||||
|
||||
(module binary racket/base
|
||||
|
@ -37,21 +36,20 @@
|
|||
#x0070000000000000
|
||||
(arithmetic-shift payload 29)))
|
||||
(dbs (integer->integer-bytes vd 8 #f #t)))
|
||||
(float (floating-point-bytes->real dbs #t 0 8)))
|
||||
(float (floating-point-bytes->real bs #t 0 4))))
|
||||
(floating-point-bytes->real dbs #t 0 8))
|
||||
(floating-point-bytes->real bs #t 0 4)))
|
||||
|
||||
(define (float->bytes v)
|
||||
(let ((v (float-value v)))
|
||||
(if (or (nan? v) (infinite? v))
|
||||
(let* ((dbs (real->floating-point-bytes v 8 #t))
|
||||
(vd (integer-bytes->integer dbs #f #t))
|
||||
(signexp (bitwise-bit-field vd 55 64))
|
||||
(payload (bitwise-bit-field vd 29 52))
|
||||
(vf (bitwise-ior (arithmetic-shift signexp 23)
|
||||
payload))
|
||||
(bs (integer->integer-bytes vf 4 #f #t)))
|
||||
bs)
|
||||
(real->floating-point-bytes v 4 #t))))
|
||||
(if (or (nan? v) (infinite? v))
|
||||
(let* ((dbs (real->floating-point-bytes v 8 #t))
|
||||
(vd (integer-bytes->integer dbs #f #t))
|
||||
(signexp (bitwise-bit-field vd 55 64))
|
||||
(payload (bitwise-bit-field vd 29 52))
|
||||
(vf (bitwise-ior (arithmetic-shift signexp 23)
|
||||
payload))
|
||||
(bs (integer->integer-bytes vf 4 #f #t)))
|
||||
bs)
|
||||
(real->floating-point-bytes v 4 #t)))
|
||||
|
||||
(define (bytes->double bs)
|
||||
(floating-point-bytes->real bs #t 0 8))
|
||||
|
|
|
@ -1,14 +0,0 @@
|
|||
#lang racket/base
|
||||
;; Wrapper struct to mark a need for 32-bit IEEE floating-point
|
||||
;; precision (de)serialization. In many circumstances, Racket lacks
|
||||
;; 32-bit floating point support, and single-flonum? always yields #f.
|
||||
|
||||
(provide (struct-out float)
|
||||
->float)
|
||||
|
||||
(struct float (value) #:transparent)
|
||||
|
||||
(define (->float v)
|
||||
(if (float? v)
|
||||
v
|
||||
(float (exact->inexact v))))
|
|
@ -8,7 +8,7 @@
|
|||
;;---------------------------------------------------------------------------
|
||||
;; Representing values
|
||||
|
||||
(require "float.rkt" "float-bytes.rkt")
|
||||
(require "float-bytes.rkt")
|
||||
(struct record (label fields) #:transparent)
|
||||
(struct annotated (annotation item) #:transparent)
|
||||
(struct embedded (value) #:transparent)
|
||||
|
@ -26,9 +26,7 @@
|
|||
[#x84 '#:end]
|
||||
[#x85 (let ((a (next))) (annotated a (next)))]
|
||||
[#x86 (embedded (next))]
|
||||
[#x87 (match (next-byte)
|
||||
[4 (bytes->float (next-bytes 4))]
|
||||
[8 (bytes->double (next-bytes 8))])]
|
||||
[#x87 (match (next-byte) [8 (bytes->double (next-bytes 8))])]
|
||||
[#xB0 (next-integer (next-varint))]
|
||||
[#xB1 (bytes->string/utf-8 (next-bytes (next-varint)))]
|
||||
[#xB2 (next-bytes (next-varint))]
|
||||
|
@ -74,7 +72,6 @@
|
|||
(match v
|
||||
[#f (write-byte #x80 out-port)]
|
||||
[#t (write-byte #x81 out-port)]
|
||||
[(float _) (write-byte #x87 out-port) (write-byte 4 out-port) (output-bytes (float->bytes v))]
|
||||
[(? flonum?) (write-byte #x87 out-port) (write-byte 8 out-port) (output-bytes (double->bytes v))]
|
||||
|
||||
[(annotated a v) (write-byte #x85 out-port) (output a) (output v)]
|
||||
|
|
|
@ -2,7 +2,6 @@
|
|||
;; Preserve, as in Fruit Preserve, as in a remarkably weak pun on pickling/dehydration etc
|
||||
|
||||
(provide (all-from-out "record.rkt")
|
||||
(all-from-out "float.rkt")
|
||||
(all-from-out "annotation.rkt")
|
||||
(all-from-out "order.rkt")
|
||||
(all-from-out "embedded.rkt")
|
||||
|
@ -31,7 +30,6 @@
|
|||
(require (only-in racket/port port->list))
|
||||
|
||||
(require "record.rkt")
|
||||
(require "float.rkt")
|
||||
(require "annotation.rkt")
|
||||
(require "order.rkt")
|
||||
(require "embedded.rkt")
|
||||
|
|
|
@ -12,7 +12,6 @@
|
|||
(require (for-syntax racket/base))
|
||||
(require "record.rkt")
|
||||
(require "annotation.rkt")
|
||||
(require "float.rkt")
|
||||
(require racket/set)
|
||||
(require racket/dict)
|
||||
(require data/order)
|
||||
|
@ -24,7 +23,7 @@
|
|||
(define (typecode v)
|
||||
(match v
|
||||
[(? boolean?) 0]
|
||||
[(or (? float?) (? single-flonum?)) 1]
|
||||
;; [(or (? float?) (? single-flonum?)) 1]
|
||||
[(? double-flonum?) 2]
|
||||
[(? integer? x) 3]
|
||||
[(? string?) 4]
|
||||
|
|
|
@ -279,7 +279,7 @@
|
|||
[(BLOCK ps ...) (grouped (convert-inner ps) " " "" "{ " " }")]
|
||||
[(SET ps ...) (grouped (convert-inner ps) " " "" "#{ " " }")]
|
||||
[(TRAILER-ANCHOR) ""]
|
||||
[(embedded v) (list "#!" (convert (encode-embedded v)))]
|
||||
[(embedded v) (list "#:" (convert (encode-embedded v)))]
|
||||
[(strip-annotations (record 'p (list s))) (symbol->string s)]
|
||||
[v (preserve->string v)]))
|
||||
|
||||
|
|
|
@ -6,7 +6,6 @@
|
|||
(require racket/match)
|
||||
(require "record.rkt")
|
||||
(require "embedded.rkt")
|
||||
(require "float.rkt")
|
||||
(require "float-bytes.rkt")
|
||||
(require "annotation.rkt")
|
||||
(require "varint.rkt")
|
||||
|
@ -76,7 +75,6 @@
|
|||
(next)))]
|
||||
[#x86 (embedded (decode-embedded (next)))]
|
||||
[#x87 (match (next-varint)
|
||||
[4 (bytes->float (next-bytes 4))]
|
||||
[8 (bytes->double (next-bytes 8))]
|
||||
[n (return (on-fail "Invalid Preserves IEEE754 size: ~v" n))])]
|
||||
[#xB0 (next-integer (next-varint))]
|
||||
|
|
|
@ -12,7 +12,6 @@
|
|||
(require racket/match)
|
||||
(require "embedded.rkt")
|
||||
(require "annotation.rkt")
|
||||
(require "float.rkt")
|
||||
(require "float-bytes.rkt")
|
||||
(require syntax/readerr)
|
||||
(require (only-in file/sha1 hex-string->bytes))
|
||||
|
@ -112,11 +111,10 @@
|
|||
[#\" (read-literal-binary)]
|
||||
[#\x (match (next-char*)
|
||||
[#\" (read-hex-binary '())]
|
||||
[#\f (read-hex-float 'float)]
|
||||
[#\d (read-hex-float 'double)]
|
||||
[#\d (read-hex-float)]
|
||||
[c (parse-error* "Invalid #x syntax: ~v" c)])]
|
||||
[#\[ (read-base64-binary '())]
|
||||
[#\! (embedded (decode-embedded (next)))]
|
||||
[#\: (embedded (decode-embedded (next)))]
|
||||
[c (on-hash c)])]
|
||||
|
||||
[c (on-char c)]))
|
||||
|
@ -242,15 +240,13 @@
|
|||
;;---------------------------------------------------------------------------
|
||||
;; Hex-encoded floating point numbers
|
||||
|
||||
(define (read-hex-float precision)
|
||||
(define (read-hex-float)
|
||||
(unless (eqv? (next-char*) #\")
|
||||
(parse-error* "Missing open-double-quote in hex-encoded floating-point number"))
|
||||
(define bs (read-hex-binary '()))
|
||||
(unless (= (bytes-length bs) (match precision ['float 4] ['double 8]))
|
||||
(unless (= (bytes-length bs) 8)
|
||||
(parse-error* "Incorrect number of bytes in hex-encoded floating-point number"))
|
||||
(match precision
|
||||
['float (bytes->float bs)]
|
||||
['double (bytes->double bs)]))
|
||||
(bytes->double bs))
|
||||
|
||||
;;---------------------------------------------------------------------------
|
||||
;; Base64-encoded ByteStrings
|
||||
|
@ -296,12 +292,8 @@
|
|||
|
||||
(define (analyze-number input)
|
||||
(match input
|
||||
[(pregexp #px"^([-+]?\\d+)(((\\.\\d+([eE][-+]?\\d+)?)|([eE][-+]?\\d+))([fF]?))?$"
|
||||
(list _ whole _ frac _ _ _ f))
|
||||
(define n (string->number (if frac (string-append whole frac) whole)))
|
||||
(cond [(not n) #f]
|
||||
[(and f (positive? (string-length f))) (float n)]
|
||||
[else n])]
|
||||
[(pregexp #px"^([-+]?\\d+)((\\.\\d+([eE][-+]?\\d+)?)|([eE][-+]?\\d+))?$" (? list?))
|
||||
(string->number input)]
|
||||
[_ #f]))
|
||||
|
||||
;;---------------------------------------------------------------------------
|
||||
|
|
|
@ -119,32 +119,6 @@
|
|||
double15: @"-sNaN" <Test #x"8708fff8000000000111" #xd"fff8000000000111">
|
||||
double16: @"+sNaN" <Test #x"87087ff8000000000001" #xd"7ff8000000000001">
|
||||
double17: @"+sNaN" <Test #x"87087ff8000000000111" #xd"7ff8000000000111">
|
||||
float0: <Test #x"870400000000" 0.0f>
|
||||
float+0: <Test #x"870400000000" +0.0f>
|
||||
float-0: <Test #x"870480000000" -0.0f>
|
||||
float1: <Test #x"87043f800000" 1.0f>
|
||||
float1u: <Test #x"87043f800000" 1.0F>
|
||||
float1a: <Test #x"87043f800000" 1e0f>
|
||||
float1b: <Test #x"87043f800000" 1.0e0f>
|
||||
float1c: <Test #x"87043f800000" 1e-0f>
|
||||
float1d: <Test #x"87043f800000" 1.0e-0f>
|
||||
float1e: <Test #x"87043f800000" 1e+0f>
|
||||
float1f: <Test #x"87043f800000" 1.0e+0f>
|
||||
float2: <Test #x"870412345678" #xf"12 34 56 78">
|
||||
float3: @"Fewer than 8 digits" <ParseError "#xf\"123456\"">
|
||||
float4: @"More than 8 digits" <ParseError "#xf\"123456789a\"">
|
||||
float5: @"Invalid chars" <ParseError "#xf\"12zz5678\"">
|
||||
float6: @"Positive infinity" <Test #x"87047f800000" #xf"7f800000">
|
||||
float7: @"Negative infinity" <Test #x"8704ff800000" #xf"ff800000">
|
||||
float8: @"+sNaN" <Test #x"87047f800001" #xf"7f800001">
|
||||
float9: @"+sNaN" <Test #x"87047f800111" #xf"7f800111">
|
||||
float10: @"-sNaN" <Test #x"8704ff800001" #xf"ff800001">
|
||||
float11: @"-sNaN" <Test #x"8704ff800111" #xf"ff800111">
|
||||
float12: @"Bad spacing" <ParseError "#xf\"12345 678\"">
|
||||
float13: @"+qNaN" <Test #x"87047fc00001" #xf"7fc00001">
|
||||
float14: @"+qNaN" <Test #x"87047fc00111" #xf"7fc00111">
|
||||
float15: @"-qNaN" <Test #x"8704ffc00001" #xf"ffc00001">
|
||||
float16: @"-qNaN" <Test #x"8704ffc00111" #xf"ffc00111">
|
||||
int-98765432109876543210987654321098765432109: <Test #x"b012feddc125aed4226c770369269596ce3f0ad3" -98765432109876543210987654321098765432109>
|
||||
int-12345678123456781234567812345678: <Test #x"b00eff642cf6684f11d1dad08c4a10b2" -12345678123456781234567812345678>
|
||||
int-1234567812345678123456781234567: <Test #x"b00df06ae570d4b4fb62ae746dce79" -1234567812345678123456781234567>
|
||||
|
@ -193,9 +167,9 @@
|
|||
list11: <Test #x"b5b0010184" [01]>
|
||||
list12: <Test #x"b5b0010c84" [12]>
|
||||
noinput0: @"No input at all" <DecodeEOF #x"">
|
||||
embed0: <Test #x"86b000" #!0>
|
||||
embed1: <Test #x"8686b000" #!#!0>
|
||||
embed2: <Test #x"b586b00086b10568656c6c6f84" [#!0 #!"hello"]>
|
||||
embed0: <Test #x"86b000" #:0>
|
||||
embed1: <Test #x"8686b000" #:#:0>
|
||||
embed2: <Test #x"b586b00086b10568656c6c6f84" [#:0 #:"hello"]>
|
||||
record1: <Test #x"b4 b30763617074757265 b4 b30764697363617264 84 84" <capture <discard>>>
|
||||
record2: <Test #x"b4 b3076f627365727665 b4 b305737065616b b4 b30764697363617264 84 b4 b30763617074757265 b4 b30764697363617264 84 84 84 84" <observe <speak <discard> <capture <discard>>>>>
|
||||
record2a: @"Commas not allowed in records" <ParseError "<observe <speak <discard>, <capture <discard>>>>">
|
||||
|
|
|
@ -7,7 +7,6 @@
|
|||
(require (only-in racket/port call-with-output-bytes))
|
||||
(require "record.rkt")
|
||||
(require "embedded.rkt")
|
||||
(require "float.rkt")
|
||||
(require "float-bytes.rkt")
|
||||
(require "annotation.rkt")
|
||||
(require "varint.rkt")
|
||||
|
@ -87,10 +86,6 @@
|
|||
[#f (output-byte #x80)]
|
||||
[#t (output-byte #x81)]
|
||||
|
||||
[(float _)
|
||||
(output-byte #x87)
|
||||
(output-byte 4)
|
||||
(output-bytes (float->bytes v))]
|
||||
[(? flonum?)
|
||||
(output-byte #x87)
|
||||
(output-byte 8)
|
||||
|
|
|
@ -11,7 +11,6 @@
|
|||
(require net/base64)
|
||||
(require "embedded.rkt")
|
||||
(require "annotation.rkt")
|
||||
(require "float.rkt")
|
||||
(require "float-bytes.rkt")
|
||||
(require "record.rkt")
|
||||
(require "object-id.rkt")
|
||||
|
@ -135,14 +134,10 @@
|
|||
(write-binary-stringlike v)
|
||||
(write-binary-base64 outer-distance v)))))
|
||||
|
||||
(define (write-float v precision)
|
||||
(define (write-float v)
|
||||
(if (or (nan? v) (infinite? v))
|
||||
(! "#x~a\"~a\""
|
||||
(match precision ['float "f"] ['double "d"])
|
||||
(bytes->hex-string (match precision
|
||||
['float (float->bytes (float v))]
|
||||
['double (double->bytes v)])))
|
||||
(! "~v~a" v (match precision ['float "f"] ['double ""]))))
|
||||
(! "#xd\"~a\"" (bytes->hex-string (double->bytes v)))
|
||||
(! "~v" v)))
|
||||
|
||||
(define (write-value distance v)
|
||||
(match v
|
||||
|
@ -155,8 +150,7 @@
|
|||
(write-value distance item)]
|
||||
[#f (! "#f")]
|
||||
[#t (! "#t")]
|
||||
[(float v) (write-float v 'float)]
|
||||
[(? flonum?) (write-float v 'double)]
|
||||
[(? flonum?) (write-float v)]
|
||||
[(? integer? x) (! "~v" v)]
|
||||
[(? string?)
|
||||
(! "\"")
|
||||
|
@ -182,7 +176,7 @@
|
|||
[(? set?) (write-sequence distance "#{" (if commas? "," "") "}" write-value (set->list v))]
|
||||
[(? dict?) (write-sequence distance "{" (if commas? "," "") "}" write-key-value (dict->list v))]
|
||||
[(embedded value)
|
||||
(! "#!")
|
||||
(! "#:")
|
||||
(write-value distance (encode-embedded value))]
|
||||
[other (error 'write-preserve/text "Attempt to serialize non-preserve: ~v" other)]))
|
||||
|
||||
|
|
|
@ -1,3 +0,0 @@
|
|||
Cargo.lock
|
||||
scratch/
|
||||
target/
|
|
@ -1,15 +0,0 @@
|
|||
[workspace]
|
||||
exclude = [
|
||||
"examples/schema-no-build",
|
||||
]
|
||||
|
||||
members = [
|
||||
"preserves",
|
||||
"preserves-path",
|
||||
"preserves-schema",
|
||||
"preserves-schema-macros",
|
||||
"preserves-tools",
|
||||
]
|
||||
|
||||
[profile.bench]
|
||||
debug = true
|
|
@ -1,41 +0,0 @@
|
|||
# Use cargo release to manage publication and versions etc.
|
||||
#
|
||||
# cargo install cargo-release
|
||||
|
||||
all:
|
||||
cargo build --all-targets
|
||||
|
||||
doc:
|
||||
cargo doc --workspace
|
||||
|
||||
x86_64-binary: x86_64-binary-release
|
||||
|
||||
x86_64-binary-release:
|
||||
cross build --target=x86_64-unknown-linux-musl --release --all-targets
|
||||
|
||||
armv7-binary: armv7-binary-release
|
||||
|
||||
armv7-binary-release:
|
||||
cross build --target=armv7-unknown-linux-musleabihf --release --all-targets
|
||||
|
||||
aarch64-binary: aarch64-binary-release
|
||||
|
||||
aarch64-binary-release:
|
||||
cross build --target=aarch64-unknown-linux-musl --release --all-targets
|
||||
|
||||
test:
|
||||
cargo test
|
||||
|
||||
test-all:
|
||||
cargo test --all-targets
|
||||
|
||||
ws-bump:
|
||||
cargo workspaces version \
|
||||
--no-global-tag \
|
||||
--individual-tag-prefix 'rust-%n@' \
|
||||
--allow-branch 'main' \
|
||||
--ignore-changes '../*'
|
||||
|
||||
ws-publish:
|
||||
cargo workspaces publish \
|
||||
--from-git
|
|
@ -0,0 +1,13 @@
|
|||
# Split out to separate repository
|
||||
|
||||
The Rust implementation of Preserves has been split out into a separate git repository,
|
||||
<https://gitlab.com/preserves/preserves-rs>.
|
||||
|
||||
The final released versions that were here were
|
||||
|
||||
- `preserves` v4.992.2,
|
||||
- `preserves-schema` v5.992.0,
|
||||
- `preserves-tools` v4.992.2, and
|
||||
- `preserves-path` v5.992.0.
|
||||
|
||||
Subsequent releases live in the other repository.
|
|
@ -1,10 +0,0 @@
|
|||
[package]
|
||||
name = "schema-no-build"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
[dependencies]
|
||||
lazy_static = "1"
|
||||
preserves = { path = "../../preserves" }
|
||||
preserves-schema = { path = "../../preserves-schema" }
|
||||
preserves-schema-macros = { path = "../../preserves-schema-macros" }
|
|
@ -1,27 +0,0 @@
|
|||
use preserves::value::IOValue;
|
||||
use preserves_schema_macros::compile_preserves_schemas;
|
||||
|
||||
compile_preserves_schemas!(
|
||||
crate::schemas,
|
||||
load("<CARGO_MANIFEST_DIR>/../../../../path/path.bin"),
|
||||
external_module(EntityRef = crate::demo_entity_ref),
|
||||
);
|
||||
|
||||
pub mod demo_entity_ref {
|
||||
use preserves::value::IOValue;
|
||||
pub type Cap = IOValue;
|
||||
}
|
||||
|
||||
preserves_schema::define_language!(language(): Language<IOValue> {
|
||||
demo: crate::schemas::Language,
|
||||
});
|
||||
|
||||
fn main() {
|
||||
use crate::schemas::path::*;
|
||||
use preserves::value::NestedValue;
|
||||
use preserves_schema::support::Unparse;
|
||||
println!("Hello, world! {:?}", (Filter::Compare {
|
||||
op: Box::new(Comparison::Eq),
|
||||
literal: IOValue::new(123),
|
||||
}).unparse(language()));
|
||||
}
|
|
@ -1,23 +0,0 @@
|
|||
[package]
|
||||
name = "preserves-path"
|
||||
version = "5.992.0"
|
||||
authors = ["Tony Garnock-Jones <tonyg@leastfixedpoint.com>"]
|
||||
edition = "2018"
|
||||
description = "Implementation of preserves-path, a query language for Preserves documents."
|
||||
homepage = "https://preserves.dev/"
|
||||
repository = "https://gitlab.com/preserves/preserves"
|
||||
license = "Apache-2.0"
|
||||
|
||||
[build-dependencies]
|
||||
preserves-schema = { path = "../preserves-schema", version = "5.992.0"}
|
||||
|
||||
[dependencies]
|
||||
preserves = { path = "../preserves", version = "4.992.1"}
|
||||
preserves-schema = { path = "../preserves-schema", version = "5.992.0"}
|
||||
|
||||
num = "0.4"
|
||||
regex = "1.5"
|
||||
thiserror = "1.0"
|
||||
|
||||
[package.metadata.workspaces]
|
||||
independent = true
|
|
@ -1,18 +0,0 @@
|
|||
use preserves_schema::compiler::*;
|
||||
|
||||
use std::io::Error;
|
||||
use std::path::PathBuf;
|
||||
|
||||
fn main() -> Result<(), Error> {
|
||||
let buildroot = PathBuf::from(std::env::var_os("OUT_DIR").unwrap());
|
||||
|
||||
let mut gen_dir = buildroot.clone();
|
||||
gen_dir.push("src/schemas");
|
||||
|
||||
let mut c = CompilerConfig::new("crate::schemas".to_owned());
|
||||
|
||||
let inputs = expand_inputs(&vec!["path.bin".to_owned()])?;
|
||||
c.load_schemas_and_bundles(&inputs, &vec![])?;
|
||||
|
||||
compile(&c, &mut CodeCollector::files(gen_dir))
|
||||
}
|
|
@ -1,7 +0,0 @@
|
|||
´³schema·³version°³definitions·³Axis´³orµµ±values´³rec´³lit³values„´³tupleµ„„„„µ±descendants´³rec´³lit³descendants„´³tupleµ„„„„µ±at´³rec´³lit³at„´³tupleµ´³named³key³any„„„„„µ±label´³rec´³lit³label„´³tupleµ„„„„µ±keys´³rec´³lit³keys„´³tupleµ„„„„µ±length´³rec´³lit³length„´³tupleµ„„„„µ±annotations´³rec´³lit³annotations„´³tupleµ„„„„µ±embedded´³rec´³lit³embedded„´³tupleµ„„„„µ±parse´³rec´³lit³parse„´³tupleµ´³named³module´³seqof´³atom³Symbol„„„´³named³name´³atom³Symbol„„„„„„µ±unparse´³rec´³lit³unparse„´³tupleµ´³named³module´³seqof´³atom³Symbol„„„´³named³name´³atom³Symbol„„„„„„„„³Step´³orµµ±Axis´³refµ„³Axis„„µ±Filter´³refµ„³Filter„„µ±Function´³refµ„³Function„„„„³Filter´³orµµ±nop´³rec´³lit³nop„´³tupleµ„„„„µ±compare´³rec´³lit³compare„´³tupleµ´³named³op´³refµ„³
|
||||
Comparison„„´³named³literal³any„„„„„µ±regex´³rec´³lit³regex„´³tupleµ´³named³regex´³atom³String„„„„„„µ±test´³rec´³lit³test„´³tupleµ´³named³pred´³refµ„³ Predicate„„„„„„µ±real´³rec´³lit³real„´³tupleµ„„„„µ±int´³rec´³lit³int„´³tupleµ„„„„µ±kind´³rec´³lit³kind„´³tupleµ´³named³kind´³refµ„³ ValueKind„„„„„„„„³Function´³rec´³lit³count„´³tupleµ´³named³selector´³refµ„³Selector„„„„„³Selector´³seqof´³refµ„³Step„„³ Predicate´³orµµ±Selector´³refµ„³Selector„„µ±not´³rec´³lit³not„´³tupleµ´³named³pred´³refµ„³ Predicate„„„„„„µ±or´³rec´³lit³or„´³tupleµ´³named³preds´³seqof´³refµ„³ Predicate„„„„„„„µ±and´³rec´³lit³and„´³tupleµ´³named³preds´³seqof´³refµ„³ Predicate„„„„„„„„„³ ValueKind´³orµµ±Boolean´³lit³Boolean„„µ±Float´³lit³Float„„µ±Double´³lit³Double„„µ±
SignedInteger´³lit³
SignedInteger„„µ±String´³lit³String„„µ±
|
||||
ByteString´³lit³
|
||||
ByteString„„µ±Symbol´³lit³Symbol„„µ±Record´³lit³Record„„µ±Sequence´³lit³Sequence„„µ±Set´³lit³Set„„µ±
|
||||
Dictionary´³lit³
|
||||
Dictionary„„µ±Embedded´³lit³Embedded„„„„³
|
||||
Comparison´³orµµ±eq´³lit³eq„„µ±ne´³lit³ne„„µ±lt´³lit³lt„„µ±ge´³lit³ge„„µ±gt´³lit³gt„„µ±le´³lit³le„„„„„³embeddedType€„„
|
|
@ -1,52 +0,0 @@
|
|||
use preserves::value::IOValue;
|
||||
|
||||
use preserves_schema::compiler::load_schema_or_bundle;
|
||||
use preserves_schema::gen::schema::Definition;
|
||||
|
||||
use std::io;
|
||||
|
||||
#[derive(Default)]
|
||||
pub struct Env(pub preserves_schema::support::interpret::Env<IOValue>);
|
||||
|
||||
pub struct Context<'a> {
|
||||
pub env: &'a Env,
|
||||
path: Vec<IOValue>,
|
||||
}
|
||||
|
||||
impl<'a> Context<'a> {
|
||||
pub fn new(env: &'a Env) -> Self {
|
||||
Context {
|
||||
env,
|
||||
path: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_path_step<R, F: FnOnce(&mut Self) -> R>(&mut self, v: &IOValue, f: F) -> R {
|
||||
self.path.push(v.clone());
|
||||
let result = f(self);
|
||||
self.path.pop();
|
||||
result
|
||||
}
|
||||
}
|
||||
|
||||
impl Env {
|
||||
pub fn new() -> Self {
|
||||
Default::default()
|
||||
}
|
||||
|
||||
pub fn load_bundle(&mut self, filename: &std::path::PathBuf) -> io::Result<bool> {
|
||||
load_schema_or_bundle(&mut self.0, filename)
|
||||
}
|
||||
|
||||
pub fn lookup_definition(
|
||||
&self,
|
||||
module: &Vec<String>,
|
||||
name: &str,
|
||||
) -> Option<&Definition<IOValue>> {
|
||||
self.0.get(module).and_then(|s| s.definitions.0.get(name))
|
||||
}
|
||||
|
||||
pub fn to_context(&self) -> Context {
|
||||
Context::new(self)
|
||||
}
|
||||
}
|
|
@ -1,17 +0,0 @@
|
|||
use std::io;
|
||||
|
||||
use thiserror::Error;
|
||||
|
||||
#[derive(Error, Debug)]
|
||||
pub enum CompilationError {
|
||||
#[error(transparent)]
|
||||
IoError(#[from] io::Error),
|
||||
#[error("Cannot mix binary operators")]
|
||||
MixedOperators,
|
||||
#[error("Invalid step")]
|
||||
InvalidStep,
|
||||
#[error("Undefined schema definition name: {0}")]
|
||||
UndefinedSchemaDefinitionName(String),
|
||||
#[error(transparent)]
|
||||
RegexError(#[from] regex::Error),
|
||||
}
|
|
@ -1,24 +0,0 @@
|
|||
pub mod context;
|
||||
pub mod error;
|
||||
pub mod parse;
|
||||
pub mod predicate;
|
||||
|
||||
pub mod schemas {
|
||||
include!(concat!(env!("OUT_DIR"), "/src/schemas/mod.rs"));
|
||||
}
|
||||
|
||||
pub mod step;
|
||||
|
||||
pub use context::Context;
|
||||
pub use context::Env;
|
||||
|
||||
pub use error::CompilationError;
|
||||
|
||||
pub use parse::parse_predicate;
|
||||
pub use parse::parse_selector;
|
||||
|
||||
pub use schemas::path::Predicate;
|
||||
pub use schemas::path::Selector;
|
||||
pub use schemas::path::Step;
|
||||
|
||||
pub use step::Node;
|
|
@ -1,323 +0,0 @@
|
|||
use crate::context::Env;
|
||||
use crate::schemas::path;
|
||||
use crate::step::Node;
|
||||
use crate::CompilationError;
|
||||
|
||||
use preserves::value::BinarySource;
|
||||
use preserves::value::BytesBinarySource;
|
||||
use preserves::value::IOValue;
|
||||
use preserves::value::NestedValue;
|
||||
use preserves::value::Reader;
|
||||
|
||||
use std::iter::Iterator;
|
||||
|
||||
#[derive(Debug)]
|
||||
enum Binop {
|
||||
Union,
|
||||
Intersection,
|
||||
}
|
||||
|
||||
fn split_values_by_symbol<'a>(tokens: &'a [IOValue], separator: &str) -> Vec<&'a [IOValue]> {
|
||||
tokens
|
||||
.split(|t| matches!(t.value().as_symbol(), Some(s) if s == separator))
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn split_binop(tokens: &[IOValue]) -> Result<(Vec<&[IOValue]>, Option<Binop>), CompilationError> {
|
||||
let union_pieces = split_values_by_symbol(&tokens, "+");
|
||||
let intersection_pieces = split_values_by_symbol(&tokens, "&");
|
||||
match (union_pieces.len(), intersection_pieces.len()) {
|
||||
(1, 1) => Ok((union_pieces, None)),
|
||||
(_, 1) => Ok((union_pieces, Some(Binop::Union))),
|
||||
(1, _) => Ok((intersection_pieces, Some(Binop::Intersection))),
|
||||
_ => Err(CompilationError::MixedOperators),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse_selector(env: &Env, tokens: &[IOValue]) -> Result<path::Selector, CompilationError> {
|
||||
let mut steps = Vec::new();
|
||||
let mut tokens = tokens;
|
||||
while let Some((s, remaining)) = parse_step(env, tokens)? {
|
||||
steps.push(s);
|
||||
tokens = remaining;
|
||||
}
|
||||
Ok(path::Selector(steps))
|
||||
}
|
||||
|
||||
pub fn parse_predicate(env: &Env, tokens: &[IOValue]) -> Result<path::Predicate, CompilationError> {
|
||||
let (pieces, binop) = split_binop(tokens)?;
|
||||
match binop {
|
||||
None => parse_non_binop(env, &pieces[0]),
|
||||
Some(o) => {
|
||||
let preds = pieces
|
||||
.into_iter()
|
||||
.map(|ts| parse_non_binop(env, &ts))
|
||||
.collect::<Result<_, _>>()?;
|
||||
Ok(match o {
|
||||
Binop::Union => path::Predicate::Or { preds },
|
||||
Binop::Intersection => path::Predicate::And { preds },
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_non_binop(env: &Env, tokens: &[IOValue]) -> Result<path::Predicate, CompilationError> {
|
||||
if !tokens.is_empty() {
|
||||
let t = tokens[0].value();
|
||||
|
||||
if let Some("!") = t.as_symbol().map(|s| s.as_str()) {
|
||||
return Ok(path::Predicate::Not {
|
||||
pred: Box::new(parse_non_binop(env, &tokens[1..])?),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
Ok(path::Predicate::Selector(Box::new(parse_selector(
|
||||
env, tokens,
|
||||
)?)))
|
||||
}
|
||||
|
||||
fn parse_schema_definition_name(
|
||||
env: &Env,
|
||||
token: &IOValue,
|
||||
) -> Result<(Vec<String>, String), CompilationError> {
|
||||
let defpath = token
|
||||
.value()
|
||||
.to_symbol()
|
||||
.map_err(|_| CompilationError::InvalidStep)?;
|
||||
let mut module: Vec<String> = defpath.split('.').map(|s| s.to_string()).collect();
|
||||
let name = module
|
||||
.pop()
|
||||
.expect("at least one element in the Schema name");
|
||||
match env.lookup_definition(&module, &name) {
|
||||
Some(_) => Ok((module, name)),
|
||||
None => Err(CompilationError::UndefinedSchemaDefinitionName(format!(
|
||||
"{:?}",
|
||||
token
|
||||
))),
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_step<'a>(
|
||||
env: &Env,
|
||||
tokens: &'a [IOValue],
|
||||
) -> Result<Option<(path::Step, &'a [IOValue])>, CompilationError> {
|
||||
if tokens.is_empty() {
|
||||
return Ok(None);
|
||||
}
|
||||
|
||||
let remainder = &tokens[1..];
|
||||
|
||||
if tokens[0].value().is_sequence() {
|
||||
return Ok(Some((
|
||||
path::Step::Filter(Box::new(path::Filter::Test {
|
||||
pred: Box::new(parse_predicate(
|
||||
env,
|
||||
tokens[0].value().as_sequence().unwrap(),
|
||||
)?),
|
||||
})),
|
||||
remainder,
|
||||
)));
|
||||
}
|
||||
|
||||
match tokens[0].value().as_record(None) {
|
||||
None => (),
|
||||
Some(r) => match r.label().value().as_symbol() {
|
||||
None => return Err(CompilationError::InvalidStep),
|
||||
Some(t) => match t.as_str() {
|
||||
"count" => {
|
||||
return Ok(Some((
|
||||
path::Step::Function(Box::new(path::Function {
|
||||
selector: parse_selector(env, r.fields())?,
|
||||
})),
|
||||
remainder,
|
||||
)))
|
||||
}
|
||||
_ => return Err(CompilationError::InvalidStep),
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
match tokens[0].value().as_symbol() {
|
||||
None => return Err(CompilationError::InvalidStep),
|
||||
Some(t) => match t.as_str() {
|
||||
"/" => Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Values)),
|
||||
remainder,
|
||||
))),
|
||||
"//" => Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Descendants)),
|
||||
remainder,
|
||||
))),
|
||||
"." => {
|
||||
let (key, remainder) = pop_step_arg(remainder)?;
|
||||
Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::At { key })),
|
||||
remainder,
|
||||
)))
|
||||
}
|
||||
".^" => Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Label)),
|
||||
remainder,
|
||||
))),
|
||||
".keys" => Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Keys)),
|
||||
remainder,
|
||||
))),
|
||||
".length" => Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Length)),
|
||||
remainder,
|
||||
))),
|
||||
".annotations" => Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Annotations)),
|
||||
remainder,
|
||||
))),
|
||||
".embedded" => Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Embedded)),
|
||||
remainder,
|
||||
))),
|
||||
"%" => {
|
||||
let (defpath, remainder) = pop_step_arg(remainder)?;
|
||||
let (module, name) = parse_schema_definition_name(env, &defpath)?;
|
||||
Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Parse { module, name })),
|
||||
remainder,
|
||||
)))
|
||||
}
|
||||
"%-" => {
|
||||
let (defpath, remainder) = pop_step_arg(remainder)?;
|
||||
let (module, name) = parse_schema_definition_name(env, &defpath)?;
|
||||
Ok(Some((
|
||||
path::Step::Axis(Box::new(path::Axis::Unparse { module, name })),
|
||||
remainder,
|
||||
)))
|
||||
}
|
||||
"*" => Ok(Some((
|
||||
path::Step::Filter(Box::new(path::Filter::Nop)),
|
||||
remainder,
|
||||
))),
|
||||
"eq" | "=" => parse_comparison(remainder, path::Comparison::Eq),
|
||||
"ne" | "!=" => parse_comparison(remainder, path::Comparison::Ne),
|
||||
"lt" => parse_comparison(remainder, path::Comparison::Lt),
|
||||
"gt" => parse_comparison(remainder, path::Comparison::Gt),
|
||||
"le" => parse_comparison(remainder, path::Comparison::Le),
|
||||
"ge" => parse_comparison(remainder, path::Comparison::Ge),
|
||||
"re" | "=r" => {
|
||||
let (regex_val, remainder) = pop_step_arg(remainder)?;
|
||||
let regex = regex_val
|
||||
.value()
|
||||
.to_string()
|
||||
.map_err(|_| CompilationError::InvalidStep)?
|
||||
.clone();
|
||||
let _ = regex::Regex::new(®ex)?;
|
||||
Ok(Some((
|
||||
path::Step::Filter(Box::new(path::Filter::Regex { regex })),
|
||||
remainder,
|
||||
)))
|
||||
}
|
||||
"^" => {
|
||||
let (literal, remainder) = pop_step_arg(remainder)?;
|
||||
Ok(Some((
|
||||
path::Step::Filter(Box::new(path::Filter::Test {
|
||||
pred: Box::new(path::Predicate::Selector(Box::new(path::Selector(vec![
|
||||
path::Step::Axis(Box::new(path::Axis::Label)),
|
||||
path::Step::Filter(Box::new(path::Filter::Compare {
|
||||
op: Box::new(path::Comparison::Eq),
|
||||
literal,
|
||||
})),
|
||||
])))),
|
||||
})),
|
||||
remainder,
|
||||
)))
|
||||
}
|
||||
|
||||
"~real" => Ok(Some((
|
||||
path::Step::Filter(Box::new(path::Filter::Real)),
|
||||
remainder,
|
||||
))),
|
||||
"~int" => Ok(Some((
|
||||
path::Step::Filter(Box::new(path::Filter::Int)),
|
||||
remainder,
|
||||
))),
|
||||
|
||||
"bool" => Ok(Some((
|
||||
path::Step::from(path::ValueKind::Boolean),
|
||||
remainder,
|
||||
))),
|
||||
"float" => Ok(Some((path::Step::from(path::ValueKind::Float), remainder))),
|
||||
"double" => Ok(Some((path::Step::from(path::ValueKind::Double), remainder))),
|
||||
"int" => Ok(Some((
|
||||
path::Step::from(path::ValueKind::SignedInteger),
|
||||
remainder,
|
||||
))),
|
||||
"string" => Ok(Some((path::Step::from(path::ValueKind::String), remainder))),
|
||||
"bytes" => Ok(Some((
|
||||
path::Step::from(path::ValueKind::ByteString),
|
||||
remainder,
|
||||
))),
|
||||
"symbol" => Ok(Some((path::Step::from(path::ValueKind::Symbol), remainder))),
|
||||
"rec" => Ok(Some((path::Step::from(path::ValueKind::Record), remainder))),
|
||||
"seq" => Ok(Some((
|
||||
path::Step::from(path::ValueKind::Sequence),
|
||||
remainder,
|
||||
))),
|
||||
"set" => Ok(Some((path::Step::from(path::ValueKind::Set), remainder))),
|
||||
"dict" => Ok(Some((
|
||||
path::Step::from(path::ValueKind::Dictionary),
|
||||
remainder,
|
||||
))),
|
||||
"embedded" => Ok(Some((
|
||||
path::Step::from(path::ValueKind::Embedded),
|
||||
remainder,
|
||||
))),
|
||||
|
||||
_ => Err(CompilationError::InvalidStep),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
impl From<path::ValueKind> for path::Step {
|
||||
fn from(k: path::ValueKind) -> Self {
|
||||
path::Step::Filter(Box::new(path::Filter::Kind { kind: Box::new(k) }))
|
||||
}
|
||||
}
|
||||
|
||||
fn pop_step_arg(tokens: &[IOValue]) -> Result<(IOValue, &[IOValue]), CompilationError> {
|
||||
if tokens.is_empty() {
|
||||
return Err(CompilationError::InvalidStep);
|
||||
}
|
||||
Ok((tokens[0].clone(), &tokens[1..]))
|
||||
}
|
||||
|
||||
fn parse_comparison(
|
||||
tokens: &[IOValue],
|
||||
op: path::Comparison,
|
||||
) -> Result<Option<(path::Step, &[IOValue])>, CompilationError> {
|
||||
let (literal, remainder) = pop_step_arg(tokens)?;
|
||||
Ok(Some((
|
||||
path::Step::Filter(Box::new(path::Filter::Compare {
|
||||
op: Box::new(op),
|
||||
literal,
|
||||
})),
|
||||
remainder,
|
||||
)))
|
||||
}
|
||||
|
||||
impl path::Selector {
|
||||
pub fn from_str(env: &Env, s: &str) -> Result<Self, CompilationError> {
|
||||
parse_selector(
|
||||
env,
|
||||
&(BytesBinarySource::new(s.as_bytes())
|
||||
.text_iovalues()
|
||||
.configured(false)
|
||||
.collect::<Result<Vec<_>, _>>()?),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
impl Node {
|
||||
pub fn from_str(env: &Env, s: &str) -> Result<Self, CompilationError> {
|
||||
let expr = path::Selector::from_str(env, s)?;
|
||||
expr.compile()
|
||||
}
|
||||
}
|
|
@ -1,68 +0,0 @@
|
|||
use crate::context::Context;
|
||||
use crate::schemas::path;
|
||||
use crate::step::BoolCollector;
|
||||
use crate::step::Node;
|
||||
use crate::step::StepMaker;
|
||||
use crate::CompilationError;
|
||||
|
||||
use preserves::value::IOValue;
|
||||
|
||||
pub trait Predicate: std::fmt::Debug {
|
||||
fn test(&mut self, ctxt: &mut Context, value: &IOValue) -> bool;
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum CompiledPredicate {
|
||||
Selector(Node),
|
||||
Not(Box<CompiledPredicate>),
|
||||
Or(Vec<CompiledPredicate>),
|
||||
And(Vec<CompiledPredicate>),
|
||||
}
|
||||
|
||||
impl path::Predicate {
|
||||
pub fn compile(&self) -> Result<CompiledPredicate, CompilationError> {
|
||||
match self {
|
||||
path::Predicate::Selector(b) => Ok(CompiledPredicate::Selector(
|
||||
(&**b).connect(BoolCollector::new())?,
|
||||
)),
|
||||
path::Predicate::Not { pred } => {
|
||||
Ok(CompiledPredicate::Not(Box::new((&**pred).compile()?)))
|
||||
}
|
||||
path::Predicate::Or { preds } => Ok(CompiledPredicate::Or(
|
||||
preds.iter().map(Self::compile).collect::<Result<_, _>>()?,
|
||||
)),
|
||||
path::Predicate::And { preds } => Ok(CompiledPredicate::And(
|
||||
preds.iter().map(Self::compile).collect::<Result<_, _>>()?,
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn exec(&self, ctxt: &mut Context, value: &IOValue) -> Result<bool, CompilationError> {
|
||||
Ok(self.compile()?.test(ctxt, value))
|
||||
}
|
||||
}
|
||||
|
||||
impl Predicate for CompiledPredicate {
|
||||
fn test(&mut self, ctxt: &mut Context, value: &IOValue) -> bool {
|
||||
match self {
|
||||
CompiledPredicate::Selector(n) => n.test(ctxt, value),
|
||||
CompiledPredicate::Not(p) => !p.test(ctxt, value),
|
||||
CompiledPredicate::Or(ps) => {
|
||||
for p in ps.iter_mut() {
|
||||
if p.test(ctxt, value) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
CompiledPredicate::And(ps) => {
|
||||
for p in ps.iter_mut() {
|
||||
if !p.test(ctxt, value) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
|
@ -1,511 +0,0 @@
|
|||
// Selectors operate on IOValues because the AST includes keys of IOValue type.
|
||||
// If we could make Schemas produce generics...
|
||||
|
||||
use crate::context::Context;
|
||||
use crate::predicate::CompiledPredicate;
|
||||
use crate::predicate::Predicate;
|
||||
use crate::schemas::path;
|
||||
use crate::CompilationError;
|
||||
|
||||
use num::bigint::BigInt;
|
||||
use num::traits::cast::FromPrimitive;
|
||||
use num::traits::cast::ToPrimitive;
|
||||
|
||||
use preserves::value::AtomClass;
|
||||
use preserves::value::CompoundClass;
|
||||
use preserves::value::IOValue;
|
||||
use preserves::value::NestedValue;
|
||||
use preserves::value::Value;
|
||||
use preserves::value::ValueClass;
|
||||
|
||||
use preserves_schema::support::interpret;
|
||||
|
||||
use std::cell::RefCell;
|
||||
use std::iter::Iterator;
|
||||
use std::rc::Rc;
|
||||
|
||||
pub trait StepMaker {
|
||||
fn connect(&self, step: Node) -> Result<Node, CompilationError>;
|
||||
}
|
||||
|
||||
pub trait Step: std::fmt::Debug {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue);
|
||||
fn finish(&mut self);
|
||||
fn reset(&mut self) -> Vec<IOValue>;
|
||||
}
|
||||
|
||||
macro_rules! delegate_finish_and_reset {
|
||||
($self:ident, $target:expr) => {
|
||||
fn finish(&mut $self) { $target.finish() }
|
||||
fn reset(&mut $self) -> Vec<IOValue> { $target.reset() }
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Node(pub Rc<RefCell<dyn Step>>);
|
||||
|
||||
#[derive(Debug)]
|
||||
struct AxisStep {
|
||||
step: Node,
|
||||
axis: path::Axis,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct CompareStep {
|
||||
op: path::Comparison,
|
||||
literal: IOValue,
|
||||
step: Node,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct RegexStep {
|
||||
regex: regex::Regex,
|
||||
step: Node,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct TestStep {
|
||||
pred: CompiledPredicate,
|
||||
step: Node,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct RealStep {
|
||||
step: Node,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct IntStep {
|
||||
step: Node,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct VecCollector {
|
||||
accumulator: Vec<IOValue>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct BoolCollector {
|
||||
seen_value: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct KindStep {
|
||||
kind: ValueClass,
|
||||
step: Node,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct CountCollector {
|
||||
count: usize,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct CountStep {
|
||||
step: Node,
|
||||
counter: Node,
|
||||
}
|
||||
|
||||
impl Node {
|
||||
fn new<S: Step + 'static>(s: S) -> Self {
|
||||
Node(Rc::new(RefCell::new(s)))
|
||||
}
|
||||
|
||||
pub fn test(&self, ctxt: &mut Context, value: &IOValue) -> bool {
|
||||
!self.exec(ctxt, value).is_empty()
|
||||
}
|
||||
|
||||
pub fn accept(&self, ctxt: &mut Context, value: &IOValue) {
|
||||
self.0.borrow_mut().accept(ctxt, value)
|
||||
}
|
||||
|
||||
pub fn finish(&self) {
|
||||
self.0.borrow_mut().finish()
|
||||
}
|
||||
|
||||
pub fn reset(&self) -> Vec<IOValue> {
|
||||
self.0.borrow_mut().reset()
|
||||
}
|
||||
|
||||
pub fn exec(&self, ctxt: &mut Context, value: &IOValue) -> Vec<IOValue> {
|
||||
self.accept(ctxt, value);
|
||||
self.finish();
|
||||
self.reset()
|
||||
}
|
||||
}
|
||||
|
||||
impl StepMaker for path::Selector {
|
||||
fn connect(&self, step: Node) -> Result<Node, CompilationError> {
|
||||
self.0.connect(step)
|
||||
}
|
||||
}
|
||||
|
||||
impl<S: StepMaker> StepMaker for Vec<S> {
|
||||
fn connect(&self, mut step: Node) -> Result<Node, CompilationError> {
|
||||
for s in self.iter().rev() {
|
||||
step = s.connect(step)?;
|
||||
}
|
||||
Ok(step)
|
||||
}
|
||||
}
|
||||
|
||||
impl StepMaker for path::Step {
|
||||
fn connect(&self, step: Node) -> Result<Node, CompilationError> {
|
||||
match self {
|
||||
path::Step::Axis(b) => (&**b).connect(step),
|
||||
path::Step::Filter(b) => (&**b).connect(step),
|
||||
path::Step::Function(b) => (&**b).connect(step),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl StepMaker for path::Axis {
|
||||
fn connect(&self, step: Node) -> Result<Node, CompilationError> {
|
||||
Ok(Node::new(AxisStep {
|
||||
step,
|
||||
axis: self.clone(),
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
fn descendants(ctxt: &mut Context, step: &mut Node, v: &IOValue) {
|
||||
step.accept(ctxt, v);
|
||||
for c in v.value().children() {
|
||||
ctxt.with_path_step(&c, |ctxt| descendants(ctxt, step, &c));
|
||||
}
|
||||
}
|
||||
|
||||
impl Step for AxisStep {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue) {
|
||||
ctxt.with_path_step(value, |ctxt| match &self.axis {
|
||||
path::Axis::Values => {
|
||||
for c in value.value().children() {
|
||||
self.step.accept(ctxt, &c)
|
||||
}
|
||||
}
|
||||
path::Axis::Descendants => descendants(ctxt, &mut self.step, value),
|
||||
path::Axis::At { key } => match value.value() {
|
||||
Value::String(s) | Value::Symbol(s) => step_index(
|
||||
ctxt,
|
||||
s.chars(),
|
||||
&key,
|
||||
|c| IOValue::new(String::from(c)),
|
||||
&mut self.step,
|
||||
),
|
||||
Value::Record(r) => {
|
||||
step_index(ctxt, r.fields().iter(), &key, |v| v.clone(), &mut self.step)
|
||||
}
|
||||
Value::Sequence(vs) => {
|
||||
step_index(ctxt, vs.iter(), &key, |v| v.clone(), &mut self.step)
|
||||
}
|
||||
Value::Dictionary(d) => {
|
||||
if let Some(v) = d.get(&key) {
|
||||
self.step.accept(ctxt, v)
|
||||
}
|
||||
}
|
||||
_ => (),
|
||||
},
|
||||
path::Axis::Label => {
|
||||
if let Some(r) = value.value().as_record(None) {
|
||||
self.step.accept(ctxt, r.label())
|
||||
}
|
||||
}
|
||||
path::Axis::Keys => match value.value() {
|
||||
Value::String(s) | Value::Symbol(s) => step_keys(ctxt, s.len(), &mut self.step),
|
||||
Value::ByteString(bs) => step_keys(ctxt, bs.len(), &mut self.step),
|
||||
Value::Record(r) => step_keys(ctxt, r.arity(), &mut self.step),
|
||||
Value::Sequence(vs) => step_keys(ctxt, vs.len(), &mut self.step),
|
||||
Value::Dictionary(d) => {
|
||||
for k in d.keys() {
|
||||
self.step.accept(ctxt, k)
|
||||
}
|
||||
}
|
||||
_ => (),
|
||||
},
|
||||
path::Axis::Length => match value.value() {
|
||||
Value::String(s) | Value::Symbol(s) => {
|
||||
self.step.accept(ctxt, &IOValue::new(s.len()))
|
||||
}
|
||||
Value::ByteString(bs) => self.step.accept(ctxt, &IOValue::new(bs.len())),
|
||||
Value::Record(r) => self.step.accept(ctxt, &IOValue::new(r.arity())),
|
||||
Value::Sequence(vs) => self.step.accept(ctxt, &IOValue::new(vs.len())),
|
||||
Value::Dictionary(d) => self.step.accept(ctxt, &IOValue::new(d.len())),
|
||||
_ => self.step.accept(ctxt, &IOValue::new(0)),
|
||||
},
|
||||
path::Axis::Annotations => {
|
||||
for c in value.annotations().slice() {
|
||||
self.step.accept(ctxt, &c)
|
||||
}
|
||||
}
|
||||
path::Axis::Embedded => {
|
||||
if let Some(d) = value.value().as_embedded() {
|
||||
self.step.accept(ctxt, d)
|
||||
}
|
||||
}
|
||||
path::Axis::Parse { module, name } => {
|
||||
if let Some(p) =
|
||||
interpret::Context::new(&ctxt.env.0).dynamic_parse(module, name, value)
|
||||
{
|
||||
self.step.accept(ctxt, &p)
|
||||
}
|
||||
}
|
||||
path::Axis::Unparse { module, name } => {
|
||||
if let Some(p) =
|
||||
interpret::Context::new(&ctxt.env.0).dynamic_unparse(module, name, value)
|
||||
{
|
||||
self.step.accept(ctxt, &p)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
delegate_finish_and_reset!(self, self.step);
|
||||
}
|
||||
|
||||
fn step_index<T, Ts: Iterator<Item = T>, F: FnOnce(T) -> IOValue>(
|
||||
ctxt: &mut Context,
|
||||
mut vs: Ts,
|
||||
key: &IOValue,
|
||||
f: F,
|
||||
step: &mut Node,
|
||||
) {
|
||||
if let Some(i) = key.value().as_usize() {
|
||||
match vs.nth(i) {
|
||||
None => (),
|
||||
Some(v) => step.accept(ctxt, &f(v)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn step_keys(ctxt: &mut Context, count: usize, step: &mut Node) {
|
||||
for i in 0..count {
|
||||
step.accept(ctxt, &IOValue::new(i))
|
||||
}
|
||||
}
|
||||
|
||||
impl StepMaker for path::Filter {
|
||||
fn connect(&self, step: Node) -> Result<Node, CompilationError> {
|
||||
match self {
|
||||
path::Filter::Nop => Ok(step),
|
||||
path::Filter::Compare { op, literal } => Ok(Node::new(CompareStep {
|
||||
op: (**op).clone(),
|
||||
literal: literal.clone(),
|
||||
step,
|
||||
})),
|
||||
path::Filter::Regex { regex } => Ok(Node::new(RegexStep {
|
||||
regex: regex::Regex::new(regex)?,
|
||||
step,
|
||||
})),
|
||||
path::Filter::Test { pred } => Ok(Node::new(TestStep {
|
||||
pred: (&**pred).compile()?,
|
||||
step,
|
||||
})),
|
||||
path::Filter::Real => Ok(Node::new(RealStep { step })),
|
||||
path::Filter::Int => Ok(Node::new(IntStep { step })),
|
||||
path::Filter::Kind { kind } => Ok(Node::new(KindStep {
|
||||
kind: match &**kind {
|
||||
path::ValueKind::Boolean => ValueClass::Atomic(AtomClass::Boolean),
|
||||
path::ValueKind::Float => ValueClass::Atomic(AtomClass::Float),
|
||||
path::ValueKind::Double => ValueClass::Atomic(AtomClass::Double),
|
||||
path::ValueKind::SignedInteger => ValueClass::Atomic(AtomClass::SignedInteger),
|
||||
path::ValueKind::String => ValueClass::Atomic(AtomClass::String),
|
||||
path::ValueKind::ByteString => ValueClass::Atomic(AtomClass::ByteString),
|
||||
path::ValueKind::Symbol => ValueClass::Atomic(AtomClass::Symbol),
|
||||
path::ValueKind::Record => ValueClass::Compound(CompoundClass::Record),
|
||||
path::ValueKind::Sequence => ValueClass::Compound(CompoundClass::Sequence),
|
||||
path::ValueKind::Set => ValueClass::Compound(CompoundClass::Set),
|
||||
path::ValueKind::Dictionary => ValueClass::Compound(CompoundClass::Dictionary),
|
||||
path::ValueKind::Embedded => ValueClass::Embedded,
|
||||
},
|
||||
step,
|
||||
})),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Step for CompareStep {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue) {
|
||||
if match self.op {
|
||||
path::Comparison::Eq => value == &self.literal,
|
||||
path::Comparison::Ne => value != &self.literal,
|
||||
path::Comparison::Lt => value < &self.literal,
|
||||
path::Comparison::Ge => value >= &self.literal,
|
||||
path::Comparison::Gt => value > &self.literal,
|
||||
path::Comparison::Le => value <= &self.literal,
|
||||
} {
|
||||
self.step.accept(ctxt, value)
|
||||
}
|
||||
}
|
||||
|
||||
delegate_finish_and_reset!(self, self.step);
|
||||
}
|
||||
|
||||
impl Step for RegexStep {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue) {
|
||||
match value.value() {
|
||||
Value::String(s) | Value::Symbol(s) => {
|
||||
if self.regex.is_match(s) {
|
||||
self.step.accept(ctxt, value)
|
||||
}
|
||||
}
|
||||
_ => (),
|
||||
}
|
||||
}
|
||||
|
||||
delegate_finish_and_reset!(self, self.step);
|
||||
}
|
||||
|
||||
impl Step for TestStep {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue) {
|
||||
if self.pred.test(ctxt, value) {
|
||||
self.step.accept(ctxt, value)
|
||||
}
|
||||
}
|
||||
|
||||
delegate_finish_and_reset!(self, self.step);
|
||||
}
|
||||
|
||||
impl Step for RealStep {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue) {
|
||||
match value.value() {
|
||||
Value::SignedInteger(i) => {
|
||||
if let Some(r) = BigInt::from(i).to_f64() {
|
||||
self.step.accept(ctxt, &IOValue::new(r))
|
||||
}
|
||||
}
|
||||
Value::Float(f) => self.step.accept(ctxt, &IOValue::new(f32::from(*f) as f64)),
|
||||
Value::Double(_) => self.step.accept(ctxt, value),
|
||||
_ => (),
|
||||
}
|
||||
}
|
||||
|
||||
delegate_finish_and_reset!(self, self.step);
|
||||
}
|
||||
|
||||
impl Step for IntStep {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue) {
|
||||
match value.value() {
|
||||
Value::SignedInteger(_) => self.step.accept(ctxt, value),
|
||||
Value::Float(f) => {
|
||||
if let Some(i) = BigInt::from_f32(f32::from(*f)) {
|
||||
self.step.accept(ctxt, &IOValue::new(i))
|
||||
}
|
||||
}
|
||||
Value::Double(d) => {
|
||||
if let Some(i) = BigInt::from_f64(f64::from(*d)) {
|
||||
self.step.accept(ctxt, &IOValue::new(i))
|
||||
}
|
||||
}
|
||||
_ => (),
|
||||
}
|
||||
}
|
||||
|
||||
delegate_finish_and_reset!(self, self.step);
|
||||
}
|
||||
|
||||
impl VecCollector {
|
||||
fn new() -> Node {
|
||||
Node::new(VecCollector {
|
||||
accumulator: Vec::new(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl Step for VecCollector {
|
||||
fn accept(&mut self, _ctxt: &mut Context, value: &IOValue) {
|
||||
self.accumulator.push(value.clone())
|
||||
}
|
||||
|
||||
fn finish(&mut self) {}
|
||||
|
||||
fn reset(&mut self) -> Vec<IOValue> {
|
||||
std::mem::take(&mut self.accumulator)
|
||||
}
|
||||
}
|
||||
|
||||
impl BoolCollector {
|
||||
pub fn new() -> Node {
|
||||
Node::new(BoolCollector { seen_value: false })
|
||||
}
|
||||
}
|
||||
|
||||
impl Step for BoolCollector {
|
||||
fn accept(&mut self, _ctxt: &mut Context, _value: &IOValue) {
|
||||
self.seen_value = true
|
||||
}
|
||||
|
||||
fn finish(&mut self) {}
|
||||
|
||||
fn reset(&mut self) -> Vec<IOValue> {
|
||||
let result = if self.seen_value {
|
||||
vec![IOValue::new(true)]
|
||||
} else {
|
||||
vec![]
|
||||
};
|
||||
self.seen_value = false;
|
||||
result
|
||||
}
|
||||
}
|
||||
|
||||
impl Step for KindStep {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue) {
|
||||
if value.value_class() == self.kind {
|
||||
self.step.accept(ctxt, value)
|
||||
}
|
||||
}
|
||||
|
||||
delegate_finish_and_reset!(self, self.step);
|
||||
}
|
||||
|
||||
impl path::Selector {
|
||||
pub fn compile(&self) -> Result<Node, CompilationError> {
|
||||
self.connect(VecCollector::new())
|
||||
}
|
||||
|
||||
pub fn exec(
|
||||
&self,
|
||||
ctxt: &mut Context,
|
||||
value: &IOValue,
|
||||
) -> Result<Vec<IOValue>, CompilationError> {
|
||||
Ok(self.compile()?.exec(ctxt, value))
|
||||
}
|
||||
}
|
||||
|
||||
impl StepMaker for path::Function {
|
||||
fn connect(&self, step: Node) -> Result<Node, CompilationError> {
|
||||
// For now, there's just one function: `count`.
|
||||
Ok(Node::new(CountStep {
|
||||
step,
|
||||
counter: self.selector.connect(CountCollector::new())?,
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
impl CountCollector {
|
||||
pub fn new() -> Node {
|
||||
Node::new(CountCollector { count: 0 })
|
||||
}
|
||||
}
|
||||
|
||||
impl Step for CountCollector {
|
||||
fn accept(&mut self, _ctxt: &mut Context, _value: &IOValue) {
|
||||
self.count += 1
|
||||
}
|
||||
|
||||
fn finish(&mut self) {}
|
||||
|
||||
fn reset(&mut self) -> Vec<IOValue> {
|
||||
let result = vec![IOValue::new(self.count)];
|
||||
self.count = 0;
|
||||
result
|
||||
}
|
||||
}
|
||||
|
||||
impl Step for CountStep {
|
||||
fn accept(&mut self, ctxt: &mut Context, value: &IOValue) {
|
||||
let count = &self.counter.exec(ctxt, value)[0];
|
||||
self.step.accept(ctxt, count)
|
||||
}
|
||||
|
||||
delegate_finish_and_reset!(self, self.step);
|
||||
}
|
|
@ -1,23 +0,0 @@
|
|||
[package]
|
||||
name = "preserves-schema-macros"
|
||||
version = "0.992.1"
|
||||
authors = ["Tony Garnock-Jones <tonyg@leastfixedpoint.com>"]
|
||||
edition = "2018"
|
||||
description = "Implementation of Preserves Schema code generation macros for Rust."
|
||||
homepage = "https://preserves.dev/"
|
||||
repository = "https://gitlab.com/preserves/preserves"
|
||||
license = "Apache-2.0"
|
||||
|
||||
[lib]
|
||||
proc-macro = true
|
||||
|
||||
[dependencies]
|
||||
preserves = { path = "../preserves", version = "4.992.1" }
|
||||
preserves-schema = { path = "../preserves-schema", version = "5.992.0" }
|
||||
|
||||
proc-macro2 = { version = "1", features = ["span-locations"] }
|
||||
quote = "1"
|
||||
syn = { version = "2", features = ["parsing", "extra-traits", "full"] }
|
||||
|
||||
[package.metadata.workspaces]
|
||||
independent = true
|
|
@ -1,227 +0,0 @@
|
|||
use preserves::value::Map;
|
||||
use preserves_schema::compiler::*;
|
||||
use preserves_schema::compiler::types::Purpose;
|
||||
use preserves_schema::gen::schema::Schema;
|
||||
use proc_macro2::Span;
|
||||
use quote::ToTokens;
|
||||
use quote::quote;
|
||||
use std::fmt::Display;
|
||||
use syn::LitStr;
|
||||
use syn::Token;
|
||||
use syn::parenthesized;
|
||||
use syn::parse::Parser;
|
||||
use syn::punctuated::Punctuated;
|
||||
|
||||
mod kw {
|
||||
use syn::custom_keyword;
|
||||
custom_keyword!(load);
|
||||
custom_keyword!(cross_reference);
|
||||
custom_keyword!(external_module);
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum Instruction {
|
||||
Namespace(String),
|
||||
Load(LitStr),
|
||||
CrossReference {
|
||||
namespace: String,
|
||||
bundle_path: LitStr,
|
||||
},
|
||||
ExternalModule {
|
||||
module_path: ModulePath,
|
||||
namespace: String,
|
||||
}
|
||||
}
|
||||
|
||||
fn syn_path_string(p: syn::Path) -> String {
|
||||
p.to_token_stream().to_string().replace(&[' ', '\t', '\n', '\r'], "")
|
||||
}
|
||||
|
||||
fn syn_litstr_resolve(s: &syn::LitStr) -> String {
|
||||
let s: String = s.value();
|
||||
match s.chars().nth(0) {
|
||||
Some('/') => s.into(),
|
||||
Some('<') => match &s[1..].split_once('>') {
|
||||
Some((envvar, remainder)) => match std::env::var(envvar) {
|
||||
Ok(p) => p + "/" + remainder,
|
||||
Err(_) => panic!("No such environment variable: {:?}", s),
|
||||
}
|
||||
None => panic!("Invalid relative path syntax: {:?}", s),
|
||||
},
|
||||
_ => panic!("Invalid path syntax: {:?}", s)
|
||||
}
|
||||
}
|
||||
|
||||
impl syn::parse::Parse for Instruction {
|
||||
fn parse(input: syn::parse::ParseStream) -> syn::Result<Self> {
|
||||
let lookahead = input.lookahead1();
|
||||
if lookahead.peek(kw::load) {
|
||||
let _: kw::load = input.parse()?;
|
||||
let content;
|
||||
let _ = parenthesized!(content in input);
|
||||
let bundle_path: syn::LitStr = content.parse()?;
|
||||
Ok(Instruction::Load(bundle_path))
|
||||
} else if lookahead.peek(kw::cross_reference) {
|
||||
let _: kw::cross_reference = input.parse()?;
|
||||
let content;
|
||||
let _ = parenthesized!(content in input);
|
||||
let namespace: syn::Path = content.parse()?;
|
||||
let _: Token![=] = content.parse()?;
|
||||
let bundle_path: syn::LitStr = content.parse()?;
|
||||
Ok(Instruction::CrossReference {
|
||||
namespace: syn_path_string(namespace),
|
||||
bundle_path,
|
||||
})
|
||||
} else if lookahead.peek(kw::external_module) {
|
||||
let _: kw::external_module = input.parse()?;
|
||||
let content;
|
||||
let _ = parenthesized!(content in input);
|
||||
let module_path = Punctuated::<syn::Ident, syn::Token![.]>::parse_separated_nonempty(&content)?;
|
||||
let _: Token![=] = content.parse()?;
|
||||
let namespace: syn::Path = content.parse()?;
|
||||
Ok(Instruction::ExternalModule {
|
||||
module_path: module_path.into_iter().map(|p| p.to_string()).collect(),
|
||||
namespace: syn_path_string(namespace),
|
||||
})
|
||||
} else {
|
||||
let ns: syn::Path = input.parse()?;
|
||||
Ok(Instruction::Namespace(syn_path_string(ns)))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct ModuleTree {
|
||||
own_body: String,
|
||||
children: Map<String, ModuleTree>,
|
||||
}
|
||||
|
||||
impl Default for ModuleTree {
|
||||
fn default() -> Self {
|
||||
ModuleTree {
|
||||
own_body: String::new(),
|
||||
children: Map::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl ModuleTree {
|
||||
fn build(outputs: Map<Option<ModulePath>, String>) -> Self {
|
||||
let mut mt = ModuleTree::default();
|
||||
for (p, c) in outputs.into_iter() {
|
||||
match p {
|
||||
None => mt.own_body = c,
|
||||
Some(k) => {
|
||||
let mut r = &mut mt;
|
||||
for e in k { r = mt.children.entry(names::render_modname(&e)).or_default(); }
|
||||
r.own_body = c;
|
||||
}
|
||||
}
|
||||
}
|
||||
mt
|
||||
}
|
||||
}
|
||||
|
||||
impl Display for ModuleTree {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(f, "{}", self.own_body)?;
|
||||
for (label, mt) in self.children.iter() {
|
||||
write!(f, "\npub mod {} {{ ", label)?;
|
||||
mt.fmt(f)?;
|
||||
write!(f, "}}")?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[proc_macro]
|
||||
pub fn compile_preserves_schemas(src: proc_macro::TokenStream) -> proc_macro::TokenStream {
|
||||
let instructions = Punctuated::<Instruction, syn::Token![,]>::parse_terminated
|
||||
.parse(src)
|
||||
.expect("valid sequence of compile_preserves_schemas instructions");
|
||||
|
||||
let mut namespace = None::<String>;
|
||||
let mut bundles_to_load = Vec::<LitStr>::new();
|
||||
let mut bundles_to_xref = Vec::<LitStr>::new();
|
||||
let mut external_modules = Vec::<ExternalModule>::new();
|
||||
|
||||
for i in instructions.into_iter() {
|
||||
match i {
|
||||
Instruction::Namespace(n) => {
|
||||
if namespace.is_some() {
|
||||
panic!("Only one namespace is permitted")
|
||||
}
|
||||
namespace = Some(n)
|
||||
}
|
||||
Instruction::Load(p) => bundles_to_load.push(p),
|
||||
Instruction::ExternalModule { module_path, namespace } => {
|
||||
external_modules.push(ExternalModule::new(module_path, &namespace));
|
||||
}
|
||||
Instruction::CrossReference { namespace, bundle_path } => {
|
||||
let mut bundle = Map::<ModulePath, Schema>::new();
|
||||
let is_schema = load_schema_or_bundle(&mut bundle, &syn_litstr_resolve(&bundle_path).into())
|
||||
.expect("Invalid schema/bundle binary");
|
||||
bundles_to_xref.push(bundle_path);
|
||||
for (k, _v) in bundle.into_iter() {
|
||||
external_modules.push(if is_schema {
|
||||
ExternalModule::new(k, &namespace)
|
||||
} else {
|
||||
let ns = namespace.clone();
|
||||
let mut pieces = vec![ns.clone()];
|
||||
pieces.extend(
|
||||
k.iter().map(|p| names::render_modname(&p)).collect::<Vec<_>>());
|
||||
ExternalModule::new(k, &pieces.join("::"))
|
||||
.set_fallback_language_types(
|
||||
move |v| vec![format!("{}::Language<{}>", ns, v)].into_iter().collect())
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let namespace = namespace.expect("Missing namespace");
|
||||
|
||||
let mut dependency_paths = Vec::<syn::LitStr>::new();
|
||||
|
||||
let mut c = CompilerConfig::new(namespace.clone());
|
||||
for b in bundles_to_load.into_iter() {
|
||||
dependency_paths.push(syn::LitStr::new(&syn_litstr_resolve(&b), Span::call_site()));
|
||||
load_schema_or_bundle_with_purpose(&mut c.bundle, &syn_litstr_resolve(&b).into(), Purpose::Codegen)
|
||||
.expect(&b.value());
|
||||
}
|
||||
for b in bundles_to_xref.into_iter() {
|
||||
dependency_paths.push(syn::LitStr::new(&syn_litstr_resolve(&b), Span::call_site()));
|
||||
load_schema_or_bundle_with_purpose(&mut c.bundle, &syn_litstr_resolve(&b).into(), Purpose::Xref)
|
||||
.expect(&b.value());
|
||||
}
|
||||
for m in external_modules.into_iter() {
|
||||
c.add_external_module(m);
|
||||
}
|
||||
|
||||
let mut outputs = Map::<Option<ModulePath>, String>::new();
|
||||
let mut collector = CodeCollector {
|
||||
emit_mod_declarations: false,
|
||||
collect_module: CodeModuleCollector::Custom {
|
||||
collect_output: &mut |p, c| {
|
||||
outputs.insert(p.cloned(), c.to_owned());
|
||||
Ok(())
|
||||
},
|
||||
},
|
||||
};
|
||||
compile(&c, &mut collector).expect("Compilation failed");
|
||||
|
||||
let top_module_source = format!(
|
||||
"pub mod {} {{ {} }}",
|
||||
names::render_modname(namespace.split("::").last().unwrap()),
|
||||
ModuleTree::build(outputs));
|
||||
let top_module: syn::Item = syn::parse_str(&top_module_source)
|
||||
.expect("Invalid generated code");
|
||||
|
||||
quote!{
|
||||
// TODO: this is ugly, but makes the code depend on the actual schema bundle:
|
||||
// See https://doc.rust-lang.org/nightly/proc_macro/tracked_path/fn.path.html
|
||||
// and https://github.com/rust-lang/rust/issues/99515
|
||||
#( const _: &'static [u8] = include_bytes!(#dependency_paths); )*
|
||||
|
||||
#top_module
|
||||
}.into()
|
||||
}
|
|
@ -1,22 +0,0 @@
|
|||
[package]
|
||||
name = "preserves-schema"
|
||||
version = "5.992.0"
|
||||
authors = ["Tony Garnock-Jones <tonyg@leastfixedpoint.com>"]
|
||||
edition = "2018"
|
||||
description = "Implementation of Preserves Schema code generation and support for Rust."
|
||||
homepage = "https://preserves.dev/"
|
||||
repository = "https://gitlab.com/preserves/preserves"
|
||||
license = "Apache-2.0"
|
||||
|
||||
[dependencies]
|
||||
preserves = { path = "../preserves", version = "4.992.1"}
|
||||
|
||||
convert_case = "0.4.0"
|
||||
glob = "0.3.0"
|
||||
lazy_static = "1.4.0"
|
||||
regex = "1.5"
|
||||
structopt = "0.3.14"
|
||||
thiserror = "1.0"
|
||||
|
||||
[package.metadata.workspaces]
|
||||
independent = true
|
|
@ -1,10 +0,0 @@
|
|||
all:
|
||||
@echo please use cargo
|
||||
|
||||
regenerate:
|
||||
cargo run -- \
|
||||
--prefix crate::gen \
|
||||
--support-crate crate \
|
||||
--rustfmt-skip \
|
||||
-o $(CURDIR)/src/gen \
|
||||
../../../schema/schema.bin
|
|
@ -1,6 +0,0 @@
|
|||
```shell
|
||||
cargo add preserves preserves-schema
|
||||
```
|
||||
|
||||
This crate ([`preserves-schema` on crates.io](https://crates.io/crates/preserves-schema)) is an
|
||||
implementation of [Preserves Schema](https://preserves.dev/preserves-schema.html) for Rust.
|
|
@ -1,112 +0,0 @@
|
|||
# Example
|
||||
|
||||
[preserves-schemac]: https://preserves.dev/doc/preserves-schemac.html
|
||||
[preserves-schema-rs]: https://preserves.dev/doc/preserves-schema-rs.html
|
||||
|
||||
Preserves schemas are written in a syntax that (ab)uses [Preserves text
|
||||
syntax][preserves::value::text] as a kind of S-expression. Schema source code looks like this:
|
||||
|
||||
```preserves-schema
|
||||
version 1 .
|
||||
Present = <Present @username string> .
|
||||
Says = <Says @who string @what string> .
|
||||
UserStatus = <Status @username string @status Status> .
|
||||
Status = =here / <away @since TimeStamp> .
|
||||
TimeStamp = string .
|
||||
```
|
||||
|
||||
Conventionally, schema source code is stored in `*.prs` files. In this example, the source code
|
||||
above is placed in `simpleChatProtocol.prs`.
|
||||
|
||||
The Rust code generator for schemas requires not source code, but instances of the [Preserves
|
||||
metaschema](https://preserves.dev/preserves-schema.html#appendix-metaschema). To compile schema
|
||||
source code to metaschema instances, use [preserves-schemac][]:
|
||||
|
||||
```shell
|
||||
yarn global add @preserves/schema
|
||||
preserves-schemac .:simpleChatProtocol.prs > simpleChatProtocol.prb
|
||||
```
|
||||
|
||||
Binary-syntax metaschema instances are conventionally stored in `*.prb` files. If you have a
|
||||
whole directory tree of `*.prs` files, you can supply just "`.`" without the "`:`"-prefixed
|
||||
fileglob part.[^converting-metaschema-to-text] See the [preserves-schemac documentation][preserves-schemac].
|
||||
|
||||
[^converting-metaschema-to-text]:
|
||||
Converting the `simpleChatProtocol.prb` file to Preserves text syntax lets us read the
|
||||
metaschema instance corresponding to the source code:
|
||||
```shell
|
||||
cat simpleChatProtocol.prb | preserves-tool convert
|
||||
```
|
||||
The result:
|
||||
```preserves
|
||||
<bundle {
|
||||
[
|
||||
simpleChatProtocol
|
||||
]: <schema {
|
||||
definitions: {
|
||||
Present: <rec <lit Present> <tuple [
|
||||
<named username <atom String>>
|
||||
]>>
|
||||
Says: <rec <lit Says> <tuple [
|
||||
<named who <atom String>>
|
||||
<named what <atom String>>
|
||||
]>>
|
||||
Status: <or [
|
||||
[
|
||||
"here"
|
||||
<lit here>
|
||||
]
|
||||
[
|
||||
"away"
|
||||
<rec <lit away> <tuple [
|
||||
<named since <ref [] TimeStamp>>
|
||||
]>>
|
||||
]
|
||||
]>
|
||||
TimeStamp: <atom String>
|
||||
UserStatus: <rec <lit Status> <tuple [
|
||||
<named username <atom String>>
|
||||
<named status <ref [] Status>>
|
||||
]>>
|
||||
}
|
||||
embeddedType: #f
|
||||
version: 1
|
||||
}>
|
||||
}>
|
||||
```
|
||||
|
||||
#### Generating Rust code from a schema
|
||||
|
||||
Generate Rust definitions corresponding to a metaschema instance with [preserves-schema-rs][].
|
||||
The best way to use it is to integrate it into your `build.rs` (see [the
|
||||
docs][preserves-schema-rs]), but you can also use it as a standalone command-line tool.
|
||||
|
||||
The following command generates a directory `./rs/chat` containing rust sources for a module
|
||||
that expects to be called `chat` in Rust code:
|
||||
|
||||
```shell
|
||||
preserves-schema-rs --output-dir rs/chat --prefix chat simpleChatProtocol.prb
|
||||
```
|
||||
|
||||
Representative excerpts from one of the generated files, `./rs/chat/simple_chat_protocol.rs`:
|
||||
|
||||
```rust,noplayground
|
||||
pub struct Present {
|
||||
pub username: std::string::String
|
||||
}
|
||||
pub struct Says {
|
||||
pub who: std::string::String,
|
||||
pub what: std::string::String
|
||||
}
|
||||
pub struct UserStatus {
|
||||
pub username: std::string::String,
|
||||
pub status: Status
|
||||
}
|
||||
pub enum Status {
|
||||
Here,
|
||||
Away {
|
||||
since: std::boxed::Box<TimeStamp>
|
||||
}
|
||||
}
|
||||
pub struct TimeStamp(pub std::string::String);
|
||||
```
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue