Tony Garnock-Jones
f88592282d
2022-02-01 15:22:30 Two problems.
- If a stop action panics (in `_terminate_facet`), the Facet is dropped before its outbound
handles are removed. With the code as it stands, this leaks assertions (!!).
- The logic for removing an outbound handle seems to be running in the wrong facet context???
(See `f.outbound_handles.remove(&handle)` in the cleanup actions
- I think I need to remove the for_myself mechanism
- and add some callbacks to run only on successful commit
2022-02-02 12:12:33 This is hard.
Here's the current implementation:
- assert
- inserts into outbound_handles of active facet
- adds cleanup action describing how to do the retraction
- enqueues the assert action, which
- calls e.assert()
- retract
- looks up & removes the cleanup action, which
- enqueues the retract action, which
- removes from outbound_handles of the WRONG facet in the WRONG actor
- calls e.retract()
- _terminate_facet
- uses outbound_handles to retract the facet's assertions
- doesn't directly touch cleanup actions, relying on retract to do that
- if one of a facet's stop actions panics, will drop the facet, leaking its assertions
- actually, even if a stop action yields `Err`, it will drop the facet and leak assertions
- yikes
- facet drop
- panics if outbound_handles is nonempty
- actor cleanup
- relies on facet tree to find assertions to retract
Revised plan:
- ✓ revise Activation/PendingEvents structures
- rename `cleanup_actions` to `outbound_assertions`
- remove `for_myself` queues and `final_actions`
- add `pre_commit_actions`, `rollback_actions` and `commit_actions`
- ✓ assert
- as before
- but on rollback, removes from `outbound_handles` (if the facet still exists) and
`outbound_assertions` (always)
- marks the new assertion as "established" on commit
- ✓ retract
- lookup in `outbound_assertions` by handle, using presence as indication it hasn't been
scheduled in this turn
- on rollback, put it back in `outbound_assertions` ONLY IF IT IS MARKED ESTABLISHED -
otherwise it is a retraction of an `assert` that has *also* been rolled back in this turn
- on commit, remove it from `outbound_handles`
- enqueue the retract action, which just calls e.retract()
- ✓ _terminate_facet
- revised quite a bit now we rely on `RunningActor::cleanup` to use `outbound_assertions`
rather than the facet tree.
- still drops Facets on panic, but this is now mostly harmless (reorders retractions a bit)
- handles `Err` from a stop action more gracefully
- slightly cleverer tracking of what needs doing based on a `TerminationDirection`
- now ONLY applies to ORDERLY cleanup of the facet tree. Disorderly cleanup ignores the
facet tree and just retracts the assertions willy-nilly.
- ✓ facet drop
- warn if outbound_handles is nonempty, but don't do anything about it
- ✓ actor cleanup
- doesn't use the facet tree at all.
- cleanly shutting down is done elsewhere
- uses the remaining entries in `outbound_assertions` (previously `cleanup_actions`) to
deal with retractions for dropped facets as well as any other facets that haven't been
cleanly shut down
- ✓ activate
- now has a panic_guard::PanicGuard RAII for conveying a crash to an actor in case the
activation is happening from a linked task or another thread (this wasn't the case in the
examples that provoked this work, though)
- simplified
- explicit commit/rollback decision
- ✓ Actor::run
- no longer uses the same path for crash-termination and success-termination
- instead, for success-termination, takes a turn that calls Activation::stop_root
- this cleans up the facet tree using _terminate_facet
- when the turn ends, it notices that the root facet is gone and shuts down the actor
- so in principle there will be nothing for actor cleanup to do
2022-02-04 13:52:34 This took days. :-(
|
||
|---|---|---|
| dev-scripts | ||
| syndicate | ||
| syndicate-macros | ||
| syndicate-server | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| Cross.toml | ||
| Makefile | ||
| README.md | ||
| rust-toolchain | ||
| syndicate-rs-server.png | ||
README.md
Syndicate/rs
A Rust implementation of:
-
the Syndicated Actor model, including assertion-based communication, failure-handling, capability-style security, dataspace entities, and facets as a structuring principle;
-
the Syndicate network protocol, including
-
a high-speed Dataspace indexing structure (
skeleton.rs; see also HOWITWORKS.md fromsyndicate-rkt) and -
a standalone Syndicate protocol "broker" service (roughly comparable in scope and intent to D-Bus); and
-
-
a handful of example programs.
The Syndicate/rs server running.
Quickstart
git clone https://git.syndicate-lang.org/syndicate-lang/syndicate-rs
cd syndicate-rs
cargo build --release
./target/release/syndicate-server -p 8001
Running the examples
In one window, start the server:
./target/release/syndicate-server -p 8001
Then, choose one of the examples below.
Producer/Consumer (sending messages)
In a second window, run a "consumer" process:
./target/release/examples/consumer
Finally, in a third window, run a "producer" process:
./target/release/examples/producer
State producer/consumer (state replication)
Replace producer with state-producer and consumer with
state-consumer, respectively, in the instructions of the previous
subsection to demonstrate Syndicate state replication.
Pingpong example (latency)
In a second window, run
./target/release/examples/pingpong pong
and in a third window, run
./target/release/examples/pingpong ping
The order is important - the difference between ping and pong is
about who kicks off the pingpong session.
Performance note
You may find better performance by restricting the server to fewer cores than you have available. For example, for me, running
taskset -c 0,1 ./target/release/syndicate-server -p 8001
roughly quadruples throughput for a single producer/consumer pair, on my 48-core AMD CPU.