forked from syndicate-lang/preserves
361 lines
14 KiB
Markdown
361 lines
14 KiB
Markdown
---
|
|
---
|
|
<link rel="stylesheet" href="preserves.css">
|
|
|
|
TODO:
|
|
|
|
- https://github.com/uwiger/sext
|
|
|
|
- http://erlang.org/doc/reference_manual/expressions.html#term-comparisons;
|
|
in particular, see the non-lexicographic ordering on tuples (vs
|
|
lists).
|
|
|
|
- should there be a built-in (i.e. recommended) reference type for external data??
|
|
- if there were, it'd give IPLD-like characteristics to the thing from the get-go
|
|
- IRIs and mime-typed things are already in there so why not content-based addressing
|
|
|
|
It is becoming VERY CLEAR that on-the-wire efficiency is... a
|
|
secondary concern. Perhaps revise the binary syntax to be less terse
|
|
and better for simple encoding and for term ordering,
|
|
canonicalization, quick indexing, etc.
|
|
|
|
- the indexing thing clashes with the term ordering thing
|
|
- maybe put the indexes at the end?? they could be optional
|
|
|
|
It might be nice to define some kind of jsonpath/xpath-like means of
|
|
naming a subterm within a Preserve. Record labels would be a kind of
|
|
assertion on the current node. Indexes and keys would be steps. It'd
|
|
be a lot like xpath I think; see also my `racket-xe` package.
|
|
|
|
- `child()` - moves into direct children
|
|
- `descendant-or-self()` - moves into direct and indirect children, including this node
|
|
- `descendant()` - moves into direct and indirect children, excluding this node
|
|
- `where[P*]` - "where" clause, applies nested path, keeping nodes with submatches
|
|
- `or[P*]` - result of first non-empty `P` match
|
|
- `at(K)` - moves into direct children whose keys are `K` from
|
|
dictionaries, sequences or records; `K` should be a number for the
|
|
latter two
|
|
- `label()` - moves into labels of records
|
|
- `equals(V)` - filters to only nodes that equal `V`
|
|
- `isa(T)` - filters to only nodes that are `T ∈
|
|
[boolean float double signed-integer string byte-string symbol record sequence set dictionary]`
|
|
|
|
Abbreviations:
|
|
|
|
/ = child()
|
|
// = descendant-or-self()
|
|
[P*] = where[P*]
|
|
Symbol = [label() equals(Symbol)]
|
|
NonSymbolAtom = at(NonSymbolAtom)
|
|
|
|
# TODO
|
|
|
|
- [DONE] allow `label[1,2,3]` and `label{a:b, c:d}`, meaning
|
|
`label([1,2,3])` and `label({a:b, c:d})`.
|
|
|
|
- explain why total order / comparison of values is important and/or useful
|
|
- what does having a total order unlock?
|
|
- explain why records are good (see below on yaml tags etc)
|
|
- hashability: comes from equivalence
|
|
- more examples
|
|
- over-8000er mountains
|
|
- yaml example from the top of https://camel.readthedocs.io/en/latest/yamlref.html
|
|
- having records with ANONYMOUS but ordered fields is good for easy
|
|
parsing in languages like C where you don't want to explicitly
|
|
search dictionaries of key/value mappings
|
|
- labels vs. yaml tags vs. annotations
|
|
- yaml tags are complex. they're relative uris, for the most part
|
|
anyway, except the local ones; they force interpretation rather
|
|
than being data, e.g. `!` forces a node to be interpreted as a
|
|
string, sequence, or map and `?` forces "tag resolution" aka
|
|
dwimming of scalar syntax. Labels here don't change how their
|
|
fields resolve at all.
|
|
- they're also used to specify particular host-language classes
|
|
and other objects.
|
|
|
|
!!python/none
|
|
!!python/bool
|
|
!!python/bytes
|
|
!!python/str
|
|
!!python/unicode
|
|
!!python/int
|
|
!!python/long
|
|
!!python/float
|
|
!!python/complex
|
|
!!python/list
|
|
!!python/tuple
|
|
!!python/dict
|
|
!!python/name:module.name
|
|
!!python/module:package.module
|
|
!!python/object:module.Cls
|
|
!!python/object/new:module.Cls
|
|
!!python/object/apply:module.f
|
|
|
|
!ruby/symbol
|
|
!ruby/sym (alias of the previous!)
|
|
!ruby/range
|
|
!ruby/regexp
|
|
!ruby/struct:StructTypeName
|
|
!ruby/object:Module::ClassName
|
|
!ruby/array:Module::ClassName (subtyping arrays! objects, not data)
|
|
!ruby/hash:Module::ClassName (subtyping hashes! objects, not data)
|
|
|
|
!perl/regexp
|
|
|
|
- yaml tag meanings are per-document or global. Labels aren't
|
|
really specified. Is this good or bad? Once there's a type
|
|
system, labels will become meaningful in a per-type context.
|
|
|
|
- yaml tags basically are meant to mean the type of the object
|
|
following. Labels are not: they are for distinguishing among
|
|
variants *within* a type. (In a unityped setting, this boils
|
|
down to the same thing at a different level; object-level vs
|
|
meta-level variants.)
|
|
|
|
- in some cases (ruby) a tag indicates a subclass: a
|
|
behavioural refinement of some *object* rather than a
|
|
structural extension of some *data*.
|
|
|
|
- yaml tags don't have intrinsic meaning: implementations are
|
|
allowed to complain if they don't recognise a tag. They also
|
|
affect how and whether an object can be used as a dict key;
|
|
labels, otoh, have intrinsic (trivial) meaning, and *any*
|
|
preserves value is allowed to be used as a dict key. YAML
|
|
documents then have implementation-specific meaning, but
|
|
Preserves have intrinsic meaning.
|
|
|
|
- yaml has schemas, holy shit, and there the tags really do
|
|
direct interpretation of values to a significant extent.
|
|
Preserves forces the application to do such interpretations:
|
|
the parser/reader won't do them for you.
|
|
- TODO: be clearer in the bit on "validity"
|
|
|
|
- yaml tags are URIs, and cannot be structured data
|
|
|
|
- annotations
|
|
- in brief: out-of-domain METADATA; implementation/metalevel, not domain/objectlevel
|
|
- comments are a good example: out-of-domain description about the
|
|
value, not part of the value itself
|
|
- uses:
|
|
- roundtripping config cf the approach taken by http://augeas.net/
|
|
- embedding trace information in messages
|
|
- provenance information
|
|
- stack information / distributed trace/continuation record
|
|
|
|
- remove comments once annotations are in!
|
|
|
|
- binary syntax: length-prefixing is good for pattern-matching,
|
|
because it allows you to reject terms based on arity without having
|
|
to scan the contents.
|
|
|
|
- hey so what about protobufs? the optional fields /
|
|
forward-and-backwards-compatibility thing is interesting.
|
|
|
|
- what about skipping e.g. lists? would need byte-length prefix
|
|
|
|
- When thinking about extensibility and forward/backward
|
|
compatibility, consider this:
|
|
<https://eighty-twenty.org/2016/09/18/gnome-flashback-patch>
|
|
|
|
- types, type-directed whitespace-sensitive parsing (oh hey it might
|
|
also lead to optimized binary parsers based on type?)
|
|
|
|
- Zephyr (here `*` is postfix Kleene star and `?` marks zero-or-one):
|
|
|
|
asdl_ty = Sum(identifier, field*, ctor, ctor*) ;; typename, common fields, at least one ctor, more ctors
|
|
| Product(identifier, field, field*) ;; ?? i guess a degenerate kind of sum??
|
|
ctor = Con(identifier, field*) ;; most like Preserves' record
|
|
field = Id(identifier, identifier?) ;; basic typename reference (?)
|
|
| Option(identifier, identifier?) ;; postfix `?`
|
|
| Sequence(identifier, identifier?) ;; postfix `*`
|
|
|
|
value = SumVal(identifier, value*, value*) ;; there are common fields
|
|
| ProductVal(value, value*)
|
|
| SequenceVal(value*)
|
|
| NoneVal
|
|
| SomeVal(value)
|
|
| PrimVal(prim)
|
|
prim = IntVal(int)
|
|
| IdentifierVal(identifier)
|
|
| StringVal(string)
|
|
|
|
- So then for us, where we have kind of union types more than
|
|
labelled sums:
|
|
- `equals(value)`, `lessthan(value)`, `greaterthan(value)`
|
|
- must be equal to / less than / greater than this value
|
|
- maybe take `range(lo,hi)` as primitive?
|
|
- no, because of infinitesimals
|
|
- `regexp(string)` ... etc? (Perhaps `pattern(regexpstring)`
|
|
is better) (Be sure to specify ECMA-262 dialect, with
|
|
restrictions a la JSON-schema
|
|
https://json-schema.org/latest/json-schema-validation.html#rfc.section.4.3)
|
|
- identifier naming a type definition
|
|
- some type definitions are builtin: `Boolean = union(equals(true), equals(false))`
|
|
- some have to be primitive rather than builtin, like
|
|
`SignedInteger` or `Double`, because they have unboundedly
|
|
(or awkwardly) many inhabitants and the class above or
|
|
below them doesn't have a limit ordinal in the right place
|
|
- parameters/`forall`?
|
|
- `record(type, type, ...)` - first one is the label type
|
|
- `list(type, ...)` - heterogeneous list of specific types
|
|
- `listof(type)` - homogeneous list
|
|
- `setof(type)`
|
|
- `{ keytype: valuetype, ... }` - heterogeneous dict
|
|
- wait, `{ keyliteral: valuetype, ... }` might be better - sugar for
|
|
`dict([equals(keyliteral), valuetype], ...)`
|
|
- `dict*(...)` for when extra members are allowed
|
|
- what about optional members?
|
|
- `dictof(keytype, valuetype)` - homogeneous dict
|
|
- `union(type, ...)`
|
|
- empty union is uninhabited type(!)
|
|
- a kind of or
|
|
- `and(type, ...)`
|
|
- simultaneous constraints on type, for range, or for range-and-type
|
|
- a kind of intersection; parallel reduction
|
|
- `interleave(type, ...)` ?? maybe, if sequences are a thing?
|
|
Could be good for organizing key-value mappings in
|
|
dictionary-brackets, because unordered... and sets...
|
|
|
|
Sketching it out:
|
|
|
|
preserves_ty =
|
|
|
|
- Oh dear, actually this is very close to being just a pattern
|
|
language without the captures.
|
|
|
|
a1.a & b1.b = a1.(a & b1.b) + b1.(a1.a & b)
|
|
|
|
- Take two.
|
|
- `=(value)`, `<(value)`, `>(value)`, `<=`, `>=`, *eq *lt *gt *le *ge
|
|
- `_` for discard, `*discard()`
|
|
- scalar values not symbols beginning with `*` match themselves as if they were `=`-wrapped
|
|
- all the special things are records, possibly 0-ary, with labels symbols starting with `*`
|
|
except for `=` etc and `_` and `...`
|
|
- if you have to match a label like `*foo` it might clash, so match `=(*foo)` instead:
|
|
`*foo(1 2 3)` ==> `=(*foo)(1 2 3)`
|
|
- `*int()` for `SignedInteger`, `*string()`, `*symbol()`,
|
|
`*bytestring()`/`*binary()`, `*float()`, `*double()`,
|
|
`*bool()`
|
|
- `*and[pattern ⋯]`
|
|
- `*or[pattern ⋯]`
|
|
- `*not(pattern)` ?
|
|
- `pattern(pattern ⋯)` - match record
|
|
- `[pattern ⋯]` - match sequence
|
|
- `#set{pattern}` - match set
|
|
- don't know how to match dictionaries yet
|
|
- view it as an interleave of its keyvalues
|
|
- `*interleave[pattern ⋯]`?
|
|
- somehow allow specification of a keyvalue that is repeating, that is optional, etc
|
|
- `{keypat:valpat ⋯ ...(keypat):...(valpat)}` ??? eww?
|
|
- `*group[pattern ⋯]` - sequence of values spliced into wider sequence?
|
|
- use literal `...` symbol (!) to mark repetition in a sequence:
|
|
`[*string() ...]`
|
|
- could use literal `?` to mark optionality; or better perhaps `*optional(pattern)`,
|
|
equivalent to `*biased-choice[pattern *group[]]`; hmm, biased choice!
|
|
- could use `*repeat(lo,hi)` or similar for counted repetition
|
|
- don't know how to write refs to other types yet! def labels starting with `*`?
|
|
|
|
*def(*foo() *or[*int() *string()]) ?
|
|
*foo()
|
|
|
|
*def(*maybe(a) *or[nothing() just(a)])
|
|
*maybe(*int())
|
|
|
|
- should those be relative URLs, or jsonpointer or something,
|
|
so can drag in types from the web?
|
|
- NOTE: No schema for indicating attachment of annotations?!?!?!
|
|
|
|
The YAML example:
|
|
|
|
database:
|
|
username: admin
|
|
password: foobar # TODO get prod passwords out of config
|
|
socket: /var/tmp/database.sock
|
|
options: {use_utf8: true}
|
|
memcached:
|
|
host: 10.0.0.99
|
|
workers:
|
|
- host: 10.0.0.101
|
|
port: 2301
|
|
- host: 10.0.0.102
|
|
port: 2302
|
|
|
|
Could be:
|
|
|
|
[ Database[Username("admin"),
|
|
@TODO("get prod passwords out of config") Password("foobar"),
|
|
Socket("/var/tmp/database.sock"),
|
|
Options[UseUTF8()]],
|
|
Memcached[Host("10.0.0.99")],
|
|
Workers[Worker("10.0.0.101", 2301),
|
|
Worker("10.0.0.102", 2302)] ]
|
|
|
|
Or
|
|
|
|
{
|
|
database: {
|
|
username: "admin",
|
|
@TODO("get prod passwords out of config")
|
|
password: "foobar",
|
|
socket: "/var/tmp/database.sock",
|
|
options: #set{use_utf8}
|
|
},
|
|
memcached: {
|
|
host: "10.0.0.99"
|
|
},
|
|
workers: [ Worker("10.0.0.101", 2301),
|
|
Worker("10.0.0.102", 2302) ]
|
|
}
|
|
|
|
Its schema-sketch could be
|
|
|
|
[ *interleave[ Database[ *interleave[ Username(*string())
|
|
Password(*string())
|
|
*optional(Socket(*string()))
|
|
*optional(Options[*option() ...]) ] ]
|
|
Memcached[ Host(*ipv4()) ... ]
|
|
Workers[ Worker(*ipv4() *u16()) ... ] ] ]
|
|
|
|
(for the first variant) or
|
|
|
|
{
|
|
database: {
|
|
username: *string(),
|
|
password: *string(),
|
|
*optional(socket): *string(),
|
|
*optional(options): #set{*option()}
|
|
},
|
|
memcached: {
|
|
host: *ipv4()
|
|
},
|
|
workers: [ Worker(*ipv4() *u16()) ... ]
|
|
}
|
|
|
|
Annotations will be allowed on any value; but also perhaps on a
|
|
key-value mapping pair?
|
|
|
|
{
|
|
@"I label the key" key: value
|
|
key @"I label the mapping": value
|
|
key: @"I label the value" value
|
|
}
|
|
|
|
??
|
|
|
|
Perhaps not.
|
|
|
|
The schema for the second YAML config sketch would allow the instance
|
|
to be written:
|
|
|
|
database:
|
|
username: admin
|
|
@TODO("get prod passwords out of config")
|
|
password: foobar
|
|
socket: /var/tmp/database.sock
|
|
options: use_utf8
|
|
memcached:
|
|
host: 10.0.0.99
|
|
workers:
|
|
Worker(10.0.0.101, 2301)
|
|
Worker(10.0.0.102, 2302)
|