## What's missing? - auth - encryption - acking and retransmission - paging out to disk - persistence of configuration - persistence of messages - statistics - exit signals (for autodeletes etc) - timestamping of messages as (an optional) part of their processing pipeline - other pluggable processing stages? how would they be configured/instantiated? ## OCaml server - Make the webserver listen on a random URL to avoid cross-site scripting attacks on localhost from malicious random internet webpages. - use lazy and Lazy.force where appropriate - Figure out how to avoid the overhead of Message.message_of_sexp - straight matching of sexps without translation gives a ~5% speedup - checking for "post" first in message_of_sexp gives a ~3% speedup - fix the Json merge operator. I don't think there can be a generally-acceptable merge, so come up with a sensible one for config perhaps? The main problem is to do with setting elements of an array; it uses `[+]` too much. ## General Cope with possibility of duplicate uuid, in e.g. queue/fanout/direct. If a subscription token matches an existing subscription, there's a collision, so choose a new token until there's no collision. SAX-style sexp reader/writer, so that we can do something sensible for enormous (e.g. gigabyte-sized) messages. The "meta" exchange probably wants to be a topic exchange. Or something. The "meta" exchange probably wants to emit how-things-are-now messages when people subscribe to it, to get them started; a kind of last-value-cache type thing. Website - http://www.flickr.com/photos/elemishra/158211069/ Switch from indirect scheduling to direct scheduling - before, on walk: 173kHz test1/test3 - after, no change. Probably because we only have a single thread active in the system when running test1/test3, so every time some work comes in, it baton-passes to the next guy in the chain. The change *would* be an improvement but we have a separate worklist (to process now) and runlist (to process after polling for events once) so it always goes back to the scheduler because the worklist is never longer than one. - Revisit once we start testing multiple concurrent streams. Extension to Sexps: - ! : binds the simplestring to the value when seen. Note that the new binding for simplestring is NOT in scope during parsing of the value, so no circular data can be constructed this way. Any previous binding is discarded *after* the value is completely read. - ? : retrieves the bound value of the simplestring. So !1:a11:hello world(?1:a1: ?1:a1: ?1:a) is equivalent to (11:hello world1: 11:hello world1: 11:hello world) And, more to the point, after a receiver has seen !1:a36:af49f5dc-0454-4ba1-9f48-a55e9c10ee35 then a simple ?1:a is all that's needed to refer to that gargantuan UUID. Another example: !1:a()!1:a(?1:a?1:a)!1:a(?1:a?1:a)?1:a is equivalent to ((()())(()())) "Having metrics around as many things as possible really helped us identify a difficult problem to diagnose." -- https://github.com/blog/767-recent-services-interruptions @DRMacIver suggests that streaming is useful in a codec, but that it should be distinct from lists; this is a really nice idea! The ideal codec, then, would support dictionaries, lists, and *streams* as a distinct kind of thing. Sexp.txt defines "text/plain; charset=iso-8859-1" as the standard MIME-type! I'm torn: ISTM either UTF-8 (if text) or application/octet-stream (if generic) should be the default, but I can see the win in having a text-like encoding that "mostly works" for display purposes. Perhaps it comes down to the fact that this is a *display* hint, not an *interpretation* hint. ... it should be an interpretation hint, then. ... (much later addition:) OTOH, iso-8859-1 is an 8-bit character set with a one-to-one mapping of characters to bytes. So it becomes possible to manipulate and transmit binary data using iso-8859-1 in ways that aren't available to utf-8. Don't forget the "five minute rule": http://cacm.acm.org/magazines/2009/7/32091-the-five-minute-rule-20-years-later/fulltext Might be nice to support, among other transports, - http (natch) - websockets - polling the file system for existence of signal files - existence = offered message - deletion = acknowledgement - maildir??