syndicate-rs/syndicate-server/src/protocol.rs

109 lines
3.5 KiB
Rust
Raw Normal View History

use futures::SinkExt;
use futures::StreamExt;
use std::future::ready;
use std::io;
use std::sync::Arc;
use syndicate::actor::*;
use syndicate::error::Error;
use syndicate::error::error;
use syndicate::relay;
2022-01-19 13:40:50 +00:00
use syndicate::trace;
use syndicate::value::NestedValue;
use tokio::net::TcpStream;
use tungstenite::Message;
struct ExitListener;
impl Entity<()> for ExitListener {
MAJOR REFACTORING OF CORE ASSERTION-TRACKING STRUCTURES. Little impact on API. Read on for details. 2022-02-01 15:22:30 Two problems. - If a stop action panics (in `_terminate_facet`), the Facet is dropped before its outbound handles are removed. With the code as it stands, this leaks assertions (!!). - The logic for removing an outbound handle seems to be running in the wrong facet context??? (See `f.outbound_handles.remove(&handle)` in the cleanup actions - I think I need to remove the for_myself mechanism - and add some callbacks to run only on successful commit 2022-02-02 12:12:33 This is hard. Here's the current implementation: - assert - inserts into outbound_handles of active facet - adds cleanup action describing how to do the retraction - enqueues the assert action, which - calls e.assert() - retract - looks up & removes the cleanup action, which - enqueues the retract action, which - removes from outbound_handles of the WRONG facet in the WRONG actor - calls e.retract() - _terminate_facet - uses outbound_handles to retract the facet's assertions - doesn't directly touch cleanup actions, relying on retract to do that - if one of a facet's stop actions panics, will drop the facet, leaking its assertions - actually, even if a stop action yields `Err`, it will drop the facet and leak assertions - yikes - facet drop - panics if outbound_handles is nonempty - actor cleanup - relies on facet tree to find assertions to retract Revised plan: - ✓ revise Activation/PendingEvents structures - rename `cleanup_actions` to `outbound_assertions` - remove `for_myself` queues and `final_actions` - add `pre_commit_actions`, `rollback_actions` and `commit_actions` - ✓ assert - as before - but on rollback, removes from `outbound_handles` (if the facet still exists) and `outbound_assertions` (always) - marks the new assertion as "established" on commit - ✓ retract - lookup in `outbound_assertions` by handle, using presence as indication it hasn't been scheduled in this turn - on rollback, put it back in `outbound_assertions` ONLY IF IT IS MARKED ESTABLISHED - otherwise it is a retraction of an `assert` that has *also* been rolled back in this turn - on commit, remove it from `outbound_handles` - enqueue the retract action, which just calls e.retract() - ✓ _terminate_facet - revised quite a bit now we rely on `RunningActor::cleanup` to use `outbound_assertions` rather than the facet tree. - still drops Facets on panic, but this is now mostly harmless (reorders retractions a bit) - handles `Err` from a stop action more gracefully - slightly cleverer tracking of what needs doing based on a `TerminationDirection` - now ONLY applies to ORDERLY cleanup of the facet tree. Disorderly cleanup ignores the facet tree and just retracts the assertions willy-nilly. - ✓ facet drop - warn if outbound_handles is nonempty, but don't do anything about it - ✓ actor cleanup - doesn't use the facet tree at all. - cleanly shutting down is done elsewhere - uses the remaining entries in `outbound_assertions` (previously `cleanup_actions`) to deal with retractions for dropped facets as well as any other facets that haven't been cleanly shut down - ✓ activate - now has a panic_guard::PanicGuard RAII for conveying a crash to an actor in case the activation is happening from a linked task or another thread (this wasn't the case in the examples that provoked this work, though) - simplified - explicit commit/rollback decision - ✓ Actor::run - no longer uses the same path for crash-termination and success-termination - instead, for success-termination, takes a turn that calls Activation::stop_root - this cleans up the facet tree using _terminate_facet - when the turn ends, it notices that the root facet is gone and shuts down the actor - so in principle there will be nothing for actor cleanup to do 2022-02-04 13:52:34 This took days. :-(
2022-02-04 12:59:37 +00:00
fn exit_hook(&mut self, _t: &mut Activation, exit_status: &Arc<ActorResult>) {
2021-08-30 10:06:40 +00:00
tracing::info!(?exit_status, "disconnect");
}
}
pub fn run_io_relay(
t: &mut Activation,
i: relay::Input,
o: relay::Output,
initial_ref: Arc<Cap>,
) -> ActorResult {
let exit_listener = t.create(ExitListener);
t.state.add_exit_hook(&exit_listener);
2021-10-05 19:11:16 +00:00
relay::TunnelRelay::run(t, i, o, Some(initial_ref), None, false);
Ok(())
}
pub fn run_connection(
2022-01-19 13:40:50 +00:00
trace_collector: Option<trace::TraceCollector>,
facet: FacetRef,
i: relay::Input,
o: relay::Output,
initial_ref: Arc<Cap>,
) {
2022-01-19 13:40:50 +00:00
let cause = trace_collector.as_ref().map(|_| trace::TurnCause::external("start-session"));
let account = Account::new(Some(AnyValue::symbol("start-session")), trace_collector);
facet.activate(&account, cause, |t| run_io_relay(t, i, o, initial_ref));
}
pub async fn detect_protocol(
2022-01-19 13:40:50 +00:00
trace_collector: Option<trace::TraceCollector>,
facet: FacetRef,
stream: TcpStream,
gateway: Arc<Cap>,
addr: std::net::SocketAddr,
) -> ActorResult {
let (i, o) = {
let mut buf = [0; 1]; // peek at the first byte to see what kind of connection to expect
match stream.peek(&mut buf).await? {
1 => match buf[0] {
b'G' /* ASCII 'G' for "GET" */ => {
2021-08-30 10:06:40 +00:00
tracing::info!(protocol = %"websocket", peer = ?addr);
let s = tokio_tungstenite::accept_async(stream).await
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
let (o, i) = s.split();
let i = i.filter_map(|r| ready(extract_binary_packets(r).transpose()));
let o = o.sink_map_err(message_error).with(|bs| ready(Ok(Message::Binary(bs))));
(relay::Input::Packets(Box::pin(i)), relay::Output::Packets(Box::pin(o)))
},
_ => {
2021-08-30 10:06:40 +00:00
tracing::info!(protocol = %"raw", peer = ?addr);
let (i, o) = stream.into_split();
(relay::Input::Bytes(Box::pin(i)),
relay::Output::Bytes(Box::pin(o /* BufWriter::new(o) */)))
}
}
0 => Err(error("closed before starting", AnyValue::new(false)))?,
_ => unreachable!()
}
};
2022-01-19 13:40:50 +00:00
run_connection(trace_collector, facet, i, o, gateway);
Ok(())
}
fn message_error<E: std::fmt::Display>(e: E) -> Error {
error(&e.to_string(), AnyValue::new(false))
}
fn extract_binary_packets(
r: Result<Message, tungstenite::Error>,
) -> Result<Option<Vec<u8>>, Error> {
match r {
Ok(m) => match m {
Message::Text(_) =>
Err("Text websocket frames are not accepted")?,
Message::Binary(bs) =>
Ok(Some(bs)),
Message::Ping(_) =>
Ok(None), // pings are handled by tungstenite before we see them
Message::Pong(_) =>
Ok(None), // unsolicited pongs are to be ignored
Message::Close(_) =>
Ok(None), // we're about to see the end of the stream, so ignore this
},
Err(e) => Err(message_error(e)),
}
}