If a facet, during X, asserts X, for all X, then X includes all
`Observe` assertions. Assertion of X should be a no-op (though
subsequent retractions of X will have no effect!) since duplicates are
ignored. However, the implementation had been ignoring whether it had
seen `Observe` assertions before, and was *always* (re)placing them
into the index, leading to runaway growth.
The repair is to only process `Observe` records on first assertion and
last retraction.
As part of this change, Dataspaces have been given names, and some
cruft from the previous implementation has been removed.
They're still there: you can use turn.state.shutdown(), which enqueues
a message for eventual actor shutdown. But it's better to use
turn.stop_root(), which terminates the actor's root facet within the
current turn, ensuring that the actor's exit_status is definitely set
by the time the turn has committed.
This is necessary to avoid a racy panic in supervision: before this
change, an asynchronous SystemMessage::Release was sent when the last
facet of an actor was stopped. Depending on load (!), any retractions
resulting from the shutdown would be delivered before the Release
arrived at the stopping actor. The supervision logic expected
exit_status to be definitely set by the time release() fired, which
wasn't always true. Now that in-turn shutdown has been implemented,
this is a reliable invariant.
A knock-on change is the need to remove
enqueue_for_myself_at_commit(), replacing it with a use of
pending.for_myself.push(). The old enqueue_for_myself_at_commit
approach could lead to lost actions as follows:
A: start linked task T, which spawns a new tokio coroutine
T: activate some facet in A and terminate A's root facet
T: at this point, A transitions to "not running"
A: spawn B, enqueuing a call to B's boot()
A: commit turn. Deliveries for others go out as usual,
but those for A will be discarded since A is "not running".
This means that the call to B's boot() goes missing.
Using pending.for_myself.push() instead assures that B's boot will
always run at the end of A's turn, without regard for whether A is in
some terminated state.
I think that this kind of race could have happened before, but
something about switching away from shutdown() seems to trigger it
somewhat reliably.
Tony Garnock-Jones