Notes and TODOs
This commit is contained in:
parent
52cd074767
commit
e6e3057de3
|
@ -0,0 +1,360 @@
|
||||||
|
---
|
||||||
|
---
|
||||||
|
<link rel="stylesheet" href="preserves.css">
|
||||||
|
|
||||||
|
TODO:
|
||||||
|
|
||||||
|
- https://github.com/uwiger/sext
|
||||||
|
|
||||||
|
- http://erlang.org/doc/reference_manual/expressions.html#term-comparisons;
|
||||||
|
in particular, see the non-lexicographic ordering on tuples (vs
|
||||||
|
lists).
|
||||||
|
|
||||||
|
- should there be a built-in (i.e. recommended) reference type for external data??
|
||||||
|
- if there were, it'd give IPLD-like characteristics to the thing from the get-go
|
||||||
|
- IRIs and mime-typed things are already in there so why not content-based addressing
|
||||||
|
|
||||||
|
It is becoming VERY CLEAR that on-the-wire efficiency is... a
|
||||||
|
secondary concern. Perhaps revise the binary syntax to be less terse
|
||||||
|
and better for simple encoding and for term ordering,
|
||||||
|
canonicalization, quick indexing, etc.
|
||||||
|
|
||||||
|
- the indexing thing clashes with the term ordering thing
|
||||||
|
- maybe put the indexes at the end?? they could be optional
|
||||||
|
|
||||||
|
It might be nice to define some kind of jsonpath/xpath-like means of
|
||||||
|
naming a subterm within a Preserve. Record labels would be a kind of
|
||||||
|
assertion on the current node. Indexes and keys would be steps. It'd
|
||||||
|
be a lot like xpath I think; see also my `racket-xe` package.
|
||||||
|
|
||||||
|
- `child()` - moves into direct children
|
||||||
|
- `descendant-or-self()` - moves into direct and indirect children, including this node
|
||||||
|
- `descendant()` - moves into direct and indirect children, excluding this node
|
||||||
|
- `where[P*]` - "where" clause, applies nested path, keeping nodes with submatches
|
||||||
|
- `or[P*]` - result of first non-empty `P` match
|
||||||
|
- `at(K)` - moves into direct children whose keys are `K` from
|
||||||
|
dictionaries, sequences or records; `K` should be a number for the
|
||||||
|
latter two
|
||||||
|
- `label()` - moves into labels of records
|
||||||
|
- `equals(V)` - filters to only nodes that equal `V`
|
||||||
|
- `isa(T)` - filters to only nodes that are `T ∈
|
||||||
|
[boolean float double signed-integer string byte-string symbol record sequence set dictionary]`
|
||||||
|
|
||||||
|
Abbreviations:
|
||||||
|
|
||||||
|
/ = child()
|
||||||
|
// = descendant-or-self()
|
||||||
|
[P*] = where[P*]
|
||||||
|
Symbol = [label() equals(Symbol)]
|
||||||
|
NonSymbolAtom = at(NonSymbolAtom)
|
||||||
|
|
||||||
|
# TODO
|
||||||
|
|
||||||
|
- [DONE] allow `label[1,2,3]` and `label{a:b, c:d}`, meaning
|
||||||
|
`label([1,2,3])` and `label({a:b, c:d})`.
|
||||||
|
|
||||||
|
- explain why total order / comparison of values is important and/or useful
|
||||||
|
- what does having a total order unlock?
|
||||||
|
- explain why records are good (see below on yaml tags etc)
|
||||||
|
- hashability: comes from equivalence
|
||||||
|
- more examples
|
||||||
|
- over-8000er mountains
|
||||||
|
- yaml example from the top of https://camel.readthedocs.io/en/latest/yamlref.html
|
||||||
|
- having records with ANONYMOUS but ordered fields is good for easy
|
||||||
|
parsing in languages like C where you don't want to explicitly
|
||||||
|
search dictionaries of key/value mappings
|
||||||
|
- labels vs. yaml tags vs. annotations
|
||||||
|
- yaml tags are complex. they're relative uris, for the most part
|
||||||
|
anyway, except the local ones; they force interpretation rather
|
||||||
|
than being data, e.g. `!` forces a node to be interpreted as a
|
||||||
|
string, sequence, or map and `?` forces "tag resolution" aka
|
||||||
|
dwimming of scalar syntax. Labels here don't change how their
|
||||||
|
fields resolve at all.
|
||||||
|
- they're also used to specify particular host-language classes
|
||||||
|
and other objects.
|
||||||
|
|
||||||
|
!!python/none
|
||||||
|
!!python/bool
|
||||||
|
!!python/bytes
|
||||||
|
!!python/str
|
||||||
|
!!python/unicode
|
||||||
|
!!python/int
|
||||||
|
!!python/long
|
||||||
|
!!python/float
|
||||||
|
!!python/complex
|
||||||
|
!!python/list
|
||||||
|
!!python/tuple
|
||||||
|
!!python/dict
|
||||||
|
!!python/name:module.name
|
||||||
|
!!python/module:package.module
|
||||||
|
!!python/object:module.Cls
|
||||||
|
!!python/object/new:module.Cls
|
||||||
|
!!python/object/apply:module.f
|
||||||
|
|
||||||
|
!ruby/symbol
|
||||||
|
!ruby/sym (alias of the previous!)
|
||||||
|
!ruby/range
|
||||||
|
!ruby/regexp
|
||||||
|
!ruby/struct:StructTypeName
|
||||||
|
!ruby/object:Module::ClassName
|
||||||
|
!ruby/array:Module::ClassName (subtyping arrays! objects, not data)
|
||||||
|
!ruby/hash:Module::ClassName (subtyping hashes! objects, not data)
|
||||||
|
|
||||||
|
!perl/regexp
|
||||||
|
|
||||||
|
- yaml tag meanings are per-document or global. Labels aren't
|
||||||
|
really specified. Is this good or bad? Once there's a type
|
||||||
|
system, labels will become meaningful in a per-type context.
|
||||||
|
|
||||||
|
- yaml tags basically are meant to mean the type of the object
|
||||||
|
following. Labels are not: they are for distinguishing among
|
||||||
|
variants *within* a type. (In a unityped setting, this boils
|
||||||
|
down to the same thing at a different level; object-level vs
|
||||||
|
meta-level variants.)
|
||||||
|
|
||||||
|
- in some cases (ruby) a tag indicates a subclass: a
|
||||||
|
behavioural refinement of some *object* rather than a
|
||||||
|
structural extension of some *data*.
|
||||||
|
|
||||||
|
- yaml tags don't have intrinsic meaning: implementations are
|
||||||
|
allowed to complain if they don't recognise a tag. They also
|
||||||
|
affect how and whether an object can be used as a dict key;
|
||||||
|
labels, otoh, have intrinsic (trivial) meaning, and *any*
|
||||||
|
preserves value is allowed to be used as a dict key. YAML
|
||||||
|
documents then have implementation-specific meaning, but
|
||||||
|
Preserves have intrinsic meaning.
|
||||||
|
|
||||||
|
- yaml has schemas, holy shit, and there the tags really do
|
||||||
|
direct interpretation of values to a significant extent.
|
||||||
|
Preserves forces the application to do such interpretations:
|
||||||
|
the parser/reader won't do them for you.
|
||||||
|
- TODO: be clearer in the bit on "validity"
|
||||||
|
|
||||||
|
- yaml tags are URIs, and cannot be structured data
|
||||||
|
|
||||||
|
- annotations
|
||||||
|
- in brief: out-of-domain METADATA; implementation/metalevel, not domain/objectlevel
|
||||||
|
- comments are a good example: out-of-domain description about the
|
||||||
|
value, not part of the value itself
|
||||||
|
- uses:
|
||||||
|
- roundtripping config cf the approach taken by http://augeas.net/
|
||||||
|
- embedding trace information in messages
|
||||||
|
- provenance information
|
||||||
|
- stack information / distributed trace/continuation record
|
||||||
|
|
||||||
|
- remove comments once annotations are in!
|
||||||
|
|
||||||
|
- binary syntax: length-prefixing is good for pattern-matching,
|
||||||
|
because it allows you to reject terms based on arity without having
|
||||||
|
to scan the contents.
|
||||||
|
|
||||||
|
- hey so what about protobufs? the optional fields /
|
||||||
|
forward-and-backwards-compatibility thing is interesting.
|
||||||
|
|
||||||
|
- what about skipping e.g. lists? would need byte-length prefix
|
||||||
|
|
||||||
|
- When thinking about extensibility and forward/backward
|
||||||
|
compatibility, consider this:
|
||||||
|
<https://eighty-twenty.org/2016/09/18/gnome-flashback-patch>
|
||||||
|
|
||||||
|
- types, type-directed whitespace-sensitive parsing (oh hey it might
|
||||||
|
also lead to optimized binary parsers based on type?)
|
||||||
|
|
||||||
|
- Zephyr (here `*` is postfix Kleene star and `?` marks zero-or-one):
|
||||||
|
|
||||||
|
asdl_ty = Sum(identifier, field*, ctor, ctor*) ;; typename, common fields, at least one ctor, more ctors
|
||||||
|
| Product(identifier, field, field*) ;; ?? i guess a degenerate kind of sum??
|
||||||
|
ctor = Con(identifier, field*) ;; most like Preserves' record
|
||||||
|
field = Id(identifier, identifier?) ;; basic typename reference (?)
|
||||||
|
| Option(identifier, identifier?) ;; postfix `?`
|
||||||
|
| Sequence(identifier, identifier?) ;; postfix `*`
|
||||||
|
|
||||||
|
value = SumVal(identifier, value*, value*) ;; there are common fields
|
||||||
|
| ProductVal(value, value*)
|
||||||
|
| SequenceVal(value*)
|
||||||
|
| NoneVal
|
||||||
|
| SomeVal(value)
|
||||||
|
| PrimVal(prim)
|
||||||
|
prim = IntVal(int)
|
||||||
|
| IdentifierVal(identifier)
|
||||||
|
| StringVal(string)
|
||||||
|
|
||||||
|
- So then for us, where we have kind of union types more than
|
||||||
|
labelled sums:
|
||||||
|
- `equals(value)`, `lessthan(value)`, `greaterthan(value)`
|
||||||
|
- must be equal to / less than / greater than this value
|
||||||
|
- maybe take `range(lo,hi)` as primitive?
|
||||||
|
- no, because of infinitesimals
|
||||||
|
- `regexp(string)` ... etc? (Perhaps `pattern(regexpstring)`
|
||||||
|
is better) (Be sure to specify ECMA-262 dialect, with
|
||||||
|
restrictions a la JSON-schema
|
||||||
|
https://json-schema.org/latest/json-schema-validation.html#rfc.section.4.3)
|
||||||
|
- identifier naming a type definition
|
||||||
|
- some type definitions are builtin: `Boolean = union(equals(true), equals(false))`
|
||||||
|
- some have to be primitive rather than builtin, like
|
||||||
|
`SignedInteger` or `Double`, because they have unboundedly
|
||||||
|
(or awkwardly) many inhabitants and the class above or
|
||||||
|
below them doesn't have a limit ordinal in the right place
|
||||||
|
- parameters/`forall`?
|
||||||
|
- `record(type, type, ...)` - first one is the label type
|
||||||
|
- `list(type, ...)` - heterogeneous list of specific types
|
||||||
|
- `listof(type)` - homogeneous list
|
||||||
|
- `setof(type)`
|
||||||
|
- `{ keytype: valuetype, ... }` - heterogeneous dict
|
||||||
|
- wait, `{ keyliteral: valuetype, ... }` might be better - sugar for
|
||||||
|
`dict([equals(keyliteral), valuetype], ...)`
|
||||||
|
- `dict*(...)` for when extra members are allowed
|
||||||
|
- what about optional members?
|
||||||
|
- `dictof(keytype, valuetype)` - homogeneous dict
|
||||||
|
- `union(type, ...)`
|
||||||
|
- empty union is uninhabited type(!)
|
||||||
|
- a kind of or
|
||||||
|
- `and(type, ...)`
|
||||||
|
- simultaneous constraints on type, for range, or for range-and-type
|
||||||
|
- a kind of intersection; parallel reduction
|
||||||
|
- `interleave(type, ...)` ?? maybe, if sequences are a thing?
|
||||||
|
Could be good for organizing key-value mappings in
|
||||||
|
dictionary-brackets, because unordered... and sets...
|
||||||
|
|
||||||
|
Sketching it out:
|
||||||
|
|
||||||
|
preserves_ty =
|
||||||
|
|
||||||
|
- Oh dear, actually this is very close to being just a pattern
|
||||||
|
language without the captures.
|
||||||
|
|
||||||
|
a1.a & b1.b = a1.(a & b1.b) + b1.(a1.a & b)
|
||||||
|
|
||||||
|
- Take two.
|
||||||
|
- `=(value)`, `<(value)`, `>(value)`, `<=`, `>=`, *eq *lt *gt *le *ge
|
||||||
|
- `_` for discard, `*discard()`
|
||||||
|
- scalar values not symbols beginning with `*` match themselves as if they were `=`-wrapped
|
||||||
|
- all the special things are records, possibly 0-ary, with labels symbols starting with `*`
|
||||||
|
except for `=` etc and `_` and `...`
|
||||||
|
- if you have to match a label like `*foo` it might clash, so match `=(*foo)` instead:
|
||||||
|
`*foo(1 2 3)` ==> `=(*foo)(1 2 3)`
|
||||||
|
- `*int()` for `SignedInteger`, `*string()`, `*symbol()`,
|
||||||
|
`*bytestring()`/`*binary()`, `*float()`, `*double()`,
|
||||||
|
`*bool()`
|
||||||
|
- `*and[pattern ⋯]`
|
||||||
|
- `*or[pattern ⋯]`
|
||||||
|
- `*not(pattern)` ?
|
||||||
|
- `pattern(pattern ⋯)` - match record
|
||||||
|
- `[pattern ⋯]` - match sequence
|
||||||
|
- `#set{pattern}` - match set
|
||||||
|
- don't know how to match dictionaries yet
|
||||||
|
- view it as an interleave of its keyvalues
|
||||||
|
- `*interleave[pattern ⋯]`?
|
||||||
|
- somehow allow specification of a keyvalue that is repeating, that is optional, etc
|
||||||
|
- `{keypat:valpat ⋯ ...(keypat):...(valpat)}` ??? eww?
|
||||||
|
- `*group[pattern ⋯]` - sequence of values spliced into wider sequence?
|
||||||
|
- use literal `...` symbol (!) to mark repetition in a sequence:
|
||||||
|
`[*string() ...]`
|
||||||
|
- could use literal `?` to mark optionality; or better perhaps `*optional(pattern)`,
|
||||||
|
equivalent to `*biased-choice[pattern *group[]]`; hmm, biased choice!
|
||||||
|
- could use `*repeat(lo,hi)` or similar for counted repetition
|
||||||
|
- don't know how to write refs to other types yet! def labels starting with `*`?
|
||||||
|
|
||||||
|
*def(*foo() *or[*int() *string()]) ?
|
||||||
|
*foo()
|
||||||
|
|
||||||
|
*def(*maybe(a) *or[nothing() just(a)])
|
||||||
|
*maybe(*int())
|
||||||
|
|
||||||
|
- should those be relative URLs, or jsonpointer or something,
|
||||||
|
so can drag in types from the web?
|
||||||
|
- NOTE: No schema for indicating attachment of annotations?!?!?!
|
||||||
|
|
||||||
|
The YAML example:
|
||||||
|
|
||||||
|
database:
|
||||||
|
username: admin
|
||||||
|
password: foobar # TODO get prod passwords out of config
|
||||||
|
socket: /var/tmp/database.sock
|
||||||
|
options: {use_utf8: true}
|
||||||
|
memcached:
|
||||||
|
host: 10.0.0.99
|
||||||
|
workers:
|
||||||
|
- host: 10.0.0.101
|
||||||
|
port: 2301
|
||||||
|
- host: 10.0.0.102
|
||||||
|
port: 2302
|
||||||
|
|
||||||
|
Could be:
|
||||||
|
|
||||||
|
[ Database[Username("admin"),
|
||||||
|
@TODO("get prod passwords out of config") Password("foobar"),
|
||||||
|
Socket("/var/tmp/database.sock"),
|
||||||
|
Options[UseUTF8()]],
|
||||||
|
Memcached[Host("10.0.0.99")],
|
||||||
|
Workers[Worker("10.0.0.101", 2301),
|
||||||
|
Worker("10.0.0.102", 2302)] ]
|
||||||
|
|
||||||
|
Or
|
||||||
|
|
||||||
|
{
|
||||||
|
database: {
|
||||||
|
username: "admin",
|
||||||
|
@TODO("get prod passwords out of config")
|
||||||
|
password: "foobar",
|
||||||
|
socket: "/var/tmp/database.sock",
|
||||||
|
options: #set{use_utf8}
|
||||||
|
},
|
||||||
|
memcached: {
|
||||||
|
host: "10.0.0.99"
|
||||||
|
},
|
||||||
|
workers: [ Worker("10.0.0.101", 2301),
|
||||||
|
Worker("10.0.0.102", 2302) ]
|
||||||
|
}
|
||||||
|
|
||||||
|
Its schema-sketch could be
|
||||||
|
|
||||||
|
[ *interleave[ Database[ *interleave[ Username(*string())
|
||||||
|
Password(*string())
|
||||||
|
*optional(Socket(*string()))
|
||||||
|
*optional(Options[*option() ...]) ] ]
|
||||||
|
Memcached[ Host(*ipv4()) ... ]
|
||||||
|
Workers[ Worker(*ipv4() *u16()) ... ] ] ]
|
||||||
|
|
||||||
|
(for the first variant) or
|
||||||
|
|
||||||
|
{
|
||||||
|
database: {
|
||||||
|
username: *string(),
|
||||||
|
password: *string(),
|
||||||
|
*optional(socket): *string(),
|
||||||
|
*optional(options): #set{*option()}
|
||||||
|
},
|
||||||
|
memcached: {
|
||||||
|
host: *ipv4()
|
||||||
|
},
|
||||||
|
workers: [ Worker(*ipv4() *u16()) ... ]
|
||||||
|
}
|
||||||
|
|
||||||
|
Annotations will be allowed on any value; but also perhaps on a
|
||||||
|
key-value mapping pair?
|
||||||
|
|
||||||
|
{
|
||||||
|
@"I label the key" key: value
|
||||||
|
key @"I label the mapping": value
|
||||||
|
key: @"I label the value" value
|
||||||
|
}
|
||||||
|
|
||||||
|
??
|
||||||
|
|
||||||
|
Perhaps not.
|
||||||
|
|
||||||
|
The schema for the second YAML config sketch would allow the instance
|
||||||
|
to be written:
|
||||||
|
|
||||||
|
database:
|
||||||
|
username: admin
|
||||||
|
@TODO("get prod passwords out of config")
|
||||||
|
password: foobar
|
||||||
|
socket: /var/tmp/database.sock
|
||||||
|
options: use_utf8
|
||||||
|
memcached:
|
||||||
|
host: 10.0.0.99
|
||||||
|
workers:
|
||||||
|
Worker(10.0.0.101, 2301)
|
||||||
|
Worker(10.0.0.102, 2302)
|
Loading…
Reference in New Issue