# Recursive Message Broker This is a *sketch* of a recursive messaging protocol, broker, and client libraries, inspired by AMQP 0-91, [PubSubHubBub](http://code.google.com/p/pubsubhubbub/), [STOMP](http://stomp.github.com/) and [reversehttp](http://reversehttp.net/). It's quite different to AMQP 1.0 (but it may be instructive to compare the two approaches). Currently, the project includes - a server (written in OCaml), with adapters for - Hop's own protocol, - a subset of AMQP 0-9-1, and - XHR streaming of messages to and from the broker. - a web-based console for the server - an OSX GUI for the server - messaging for the web - client libraries for various languages (Java, Racket, Javascript) ## A sketch? Honestly, not meant to be production software... yet. ## Background Messaging à la AMQP 0-91 can be broken down into a few core pieces: - transmission and receipt of messages (publishes, deliveries, and gets) - subscription management (subsuming enrollment, bindings, consumers and relays) - directory (naming of resources in the broker) - object management (creation and destruction of remote resources) AMQP itself, being a first mover in its space, isn't as orthogonal as it could be. It can be greatly simplified without losing anything of value. This experiment is intended to demonstrate one possible way of paring each of the core pieces of AMQP-style messaging back to their essences. ### More detail TBD. - what recursive means in this context - doing things this way gives you shovels (relays) for free - and effortless interop with legacy messaging networks (including UDP, SMTP, IMAP, HTTP etc) - and effortless federation - and a big step closer to a sensible semantics for transactions - relays (including active *client* connections!) are just nodes in the network, addressable like any other - so `(post! somerelay (post! someremotenode ...))` and so on is the way to cause things to happen remotely. ## Compiling the server The server is written in [OCaml](http://caml.inria.fr/). To build and run the server, you will need: - [OCaml itself](http://caml.inria.fr/download.en.html), version 3.12 or newer - [OCaml Findlib](http://projects.camlcity.org/projects/findlib.html); I have used 1.2.8 and 1.3.1, but older versions may well work - [libev](http://software.schmorp.de/pkg/libev.html) installed somewhere that Findlib can find it - [python](http://www.python.org/) to generate parts of the protocol codecs Make sure you have `ocamlopt`, `ocamlbuild`, `ocamlfind` etc. on your path. Then, in the `server` subdirectory, run `make`. It should first compile [Lwt](http://ocsigen.org/lwt/), which is included as a third-party library, and then should proceed to compiling the server itself. If `ocamlfind` can't find `libev`, try setting (and exporting) the environment variables `C_INCLUDE_PATH` and `LIBRARY_PATH` to point to the include and lib directories containing `libev`'s files. To run the server, simply run `./server/hop_server.native`, or just `make run` from within the `server` directory. ## Working with the management and monitoring webpages If you want to edit and/or recompile the server's built-in webpages, you will need to have installed - [xsltproc](http://xmlsoft.org/xslt/xsltproc2.html) to make the webpages from the templates - [recess](http://twitter.github.com/recess/) to compile the LESS into CSS ## Compiling the Java client library You will need a recent JDK, and Ant v1.6 or newer. Change to the `java` subdirectory, and run `ant`. You will end up with a file `build/lib/hop.jar`, which contains the client library and some test programs. ## Run it Open three terminals. Run the server in one of them. You should see output like the following: hop ALPHA, Copyright (C) 2012 Tony Garnock-Jones. This program comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the GNU General Public License (version 3 or later) for details. info: ("Node bound" "factory" "factory") info: ("Registered node class" "queue") info: ("Registered node class" "fanout") info: ("Registered node class" "direct") info: ("Node bound" "meta" "direct") info: ("Node create ok" "direct" ("meta") "" "" "meta") info: ("Node bound" "amq.direct" "direct") info: ("Node create ok" "direct" ("amq.direct") "" "" "amq.direct") info: ("Node bound" "amq.fanout" "fanout") info: ("Node create ok" "fanout" ("amq.fanout") "" "" "amq.fanout") info: ("Accepting connections" "AMQP" "5672") info: ("Accepting connections" "HTTP" "5678") info: ("Accepting connections" "Hop" "5671") info: ("Waiting for milestone" "AMQP ready") info: ("Achieved milestone" "AMQP ready") info: ("Waiting for milestone" "HTTP ready") info: ("Achieved milestone" "HTTP ready") info: ("Waiting for milestone" "Hop ready") info: ("Achieved milestone" "Hop ready") info: ("Achieved milestone" "Server initialized") In the second terminal, run the consuming half of the Java test program pair: java -cp hop.jar hop.Test1 localhost In the third, run the producing half: java -cp hop.jar hop.Test3 localhost ## Wire protocol Obviously the wire protocol itself here is the simplest thing that could possibly work, and you'd never use anything like this inefficient in a real system. That said, this is what's there right now: ### Message transfer `(post )` - Instructs the receiving node to route (or process) the given `message` according to the given `routing-key`. Different kinds of nodes will do different things here, and in particular, will interpret the routing key differently. Queues, for example, will ignore the routing key and will deliver the message to only one of their active subscribers, whereas exchanges will generally match the routing key against their active subscriptions and will deliver the message on to all matches. ### Subscription management `(subscribe )` - Instructs the receiving node to create a new subscription. The new subscription will only route messages matching the `routing-key-filter`, which is interpreted on a per-node-type basis as above for `routing-key`. Matching messages will be sent to `target-node` using `post!`, with a routing key of `target-routing-key`. The `reply-node` parameter, if nonempty, instructs the receiving node to send confirmation of subscription (along with a token that can be used with `unsubscribe` below) to the given address and routing key. If `reply-node` is empty, no confirmation of subscription is sent. `(unsubscribe )` - Instructs the receiving node to delete a previously established subscription. The `token` comes from the `subscribe-ok` message sent to `reply-node` after a successful `subscribe` operation. ### Object management `(create )` - Instructs the receiving object factory node to construct a new instance of the given `class-name`, with the given `argument` supplied to the constructor. The `reply-node` and `reply-routing-key` are used to send confirmation of completion to some waiting node. ## Copyright and licensing Hop is Copyright 2010, 2011, 2012 Tony Garnock-Jones . Hop is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. Hop is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Hop. If not, see .