diff --git a/README.md b/README.md new file mode 100644 index 0000000..a496657 --- /dev/null +++ b/README.md @@ -0,0 +1,190 @@ +# Recursive Message Broker + +This is a *sketch* of a recursive messaging protocol, broker, and +client libraries, inspired by AMQP 0-91, +[PubSubHubBub](http://code.google.com/p/pubsubhubbub/), +[STOMP](http://stomp.github.com/) and +[reversehttp](http://reversehttp.net/). It's quite different to AMQP +1.0 (but it may be instructive to compare the two approaches). + +Currently, the project includes + + - a server (written in OCaml), with adapters for + - Hop's own protocol, + - a subset of AMQP 0-9-1, and + - XHR streaming of messages to and from the broker. + - a web-based console for the server + - an OSX GUI for the server + - messaging for the web + - client libraries for various languages (Java, Racket, Javascript) + +## A sketch? + +Honestly, not meant to be production software... yet. + +## Background + +Messaging à la AMQP 0-91 can be broken down into a few core pieces: + + - transmission and receipt of messages (publishes, deliveries, and gets) + + - subscription management (subsuming enrollment, bindings, consumers and relays) + + - directory (naming of resources in the broker) + + - object management (creation and destruction of remote resources) + +AMQP itself, being a first mover in its space, isn't as orthogonal as +it could be. It can be greatly simplified without losing anything of +value. This experiment is intended to demonstrate one possible way of +paring each of the core pieces of AMQP-style messaging back to their +essences. + +### More detail + +TBD. + + - what recursive means in this context + - doing things this way gives you shovels (relays) for free + - and effortless interop with legacy messaging networks (including UDP, SMTP, IMAP, HTTP etc) + - and effortless federation + - and a big step closer to a sensible semantics for transactions + + - relays (including active *client* connections!) are just nodes in + the network, addressable like any other - so `(post! somerelay + (post! someremotenode ...))` and so on is the way to cause things + to happen remotely. + +## Compiling the server + +The server is written in [OCaml](http://caml.inria.fr/). To build and +run the server, you will need: + + - [OCaml itself](http://caml.inria.fr/download.en.html), version 3.12 or newer + - [OCaml Findlib](http://projects.camlcity.org/projects/findlib.html); I have used 1.2.8 and 1.3.1, but older versions may well work + +Make sure you have `ocamlopt`, `ocamlbuild`, `ocamlfind` etc. on your +path. Then, in the `server` subdirectory, run `make`. It should first +compile [Lwt](http://ocsigen.org/lwt/), which is included as a +third-party library, and then should proceed to compiling the server +itself. + +To run the server, simply run `./server/hop_server.native`, or just +`make run` from within the `server` directory. + +## Working with the management and monitoring webpages + +If you want to edit and/or recompile the server's built-in webpages, +you will need to have installed + + - [xsltproc](http://xmlsoft.org/xslt/xsltproc2.html) to make the webpages from the templates + - [recess](http://twitter.github.com/recess/) to compile the LESS into CSS + +## Compiling the Java client library + +You will need a recent JDK, and Ant v1.6 or newer. Change to the +`java` subdirectory, and run `ant`. You will end up with a file +`build/lib/hop.jar`, which contains the client library and some test +programs. + +## Run it + +Open three terminals. Run the server in one of them. You should see +output like the following: + + hop ALPHA, Copyright (C) 2012 Tony Garnock-Jones. + This program comes with ABSOLUTELY NO WARRANTY. This is free software, + and you are welcome to redistribute it under certain conditions. + See the GNU General Public License (version 3 or later) for details. + info: ("Node bound" "factory" "factory") + info: ("Registered node class" "queue") + info: ("Registered node class" "fanout") + info: ("Registered node class" "direct") + info: ("Node bound" "meta" "direct") + info: ("Node create ok" "direct" ("meta") "" "" "meta") + info: ("Node bound" "amq.direct" "direct") + info: ("Node create ok" "direct" ("amq.direct") "" "" "amq.direct") + info: ("Node bound" "amq.fanout" "fanout") + info: ("Node create ok" "fanout" ("amq.fanout") "" "" "amq.fanout") + info: ("Accepting connections" "AMQP" "5672") + info: ("Accepting connections" "HTTP" "5678") + info: ("Accepting connections" "Hop" "5671") + info: ("Waiting for milestone" "AMQP ready") + info: ("Achieved milestone" "AMQP ready") + info: ("Waiting for milestone" "HTTP ready") + info: ("Achieved milestone" "HTTP ready") + info: ("Waiting for milestone" "Hop ready") + info: ("Achieved milestone" "Hop ready") + info: ("Achieved milestone" "Server initialized") + +In the second terminal, run the consuming half of the Java test +program pair: + + java -cp hop.jar hop.Test1 localhost + +In the third, run the producing half: + + java -cp hop.jar hop.Test3 localhost + +## Wire protocol + +Obviously the wire protocol itself here is the simplest thing that +could possibly work, and you'd never use anything like this +inefficient in a real system. That said, this is what's there right +now: + +### Message transfer + +`(post )` - Instructs the +receiving node to route (or process) the given `message` according to +the given `routing-key`. Different kinds of nodes will do different +things here, and in particular, will interpret the routing key +differently. Queues, for example, will ignore the routing key and will +deliver the message to only one of their active subscribers, whereas +exchanges will generally match the routing key against their active +subscriptions and will deliver the message on to all matches. + +### Subscription management + +`(subscribe + )` - Instructs the receiving node to +create a new subscription. The new subscription will only route +messages matching the `routing-key-filter`, which is interpreted on a +per-node-type basis as above for `routing-key`. Matching messages will +be sent to `target-node` using `post!`, with a routing key of +`target-routing-key`. The `reply-node` parameter, if nonempty, +instructs the receiving node to send confirmation of subscription +(along with a token that can be used with `unsubscribe` below) to the +given address and routing key. If `reply-node` is empty, no +confirmation of subscription is sent. + +`(unsubscribe )` - Instructs the receiving node to delete a +previously established subscription. The `token` comes from the +`subscribe-ok` message sent to `reply-node` after a successful +`subscribe` operation. + +### Object management + +`(create )` - +Instructs the receiving object factory node to construct a new +instance of the given `class-name`, with the given `argument` supplied +to the constructor. The `reply-node` and `reply-routing-key` are used +to send confirmation of completion to some waiting node. + +## Copyright and licensing + +Hop is Copyright 2010, 2011, 2012 Tony Garnock-Jones +. + +Hop is free software: you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation, either version 3 of the License, or (at your +option) any later version. + +Hop is distributed in the hope that it will be useful, but WITHOUT +ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +for more details. + +You should have received a copy of the GNU General Public License +along with Hop. If not, see .