# Recursive Message Broker

This is a *sketch* of a recursive messaging protocol, broker, and
client libraries, inspired by AMQP 0-91,
[PubSubHubBub](http://code.google.com/p/pubsubhubbub/),
[STOMP](http://stomp.github.com/) and
[reversehttp](http://reversehttp.net/). It's quite different to AMQP
1.0 (but it may be instructive to compare the two approaches).

Currently, the project includes

 - a server (written in OCaml), with adapters for
    - Hop's own protocol,
    - a subset of AMQP 0-9-1, and
    - XHR streaming of messages to and from the broker.
 - a web-based console for the server
 - an OSX GUI for the server
 - messaging for the web
 - client libraries for various languages (Java, Racket, Javascript)

## A sketch?

Honestly, not meant to be production software... yet.

## Background

Messaging à la AMQP 0-91 can be broken down into a few core pieces:

 - transmission and receipt of messages (publishes, deliveries, and gets)

 - subscription management (subsuming enrollment, bindings, consumers and relays)

 - directory (naming of resources in the broker)

 - object management (creation and destruction of remote resources)

AMQP itself, being a first mover in its space, isn't as orthogonal as
it could be. It can be greatly simplified without losing anything of
value. This experiment is intended to demonstrate one possible way of
paring each of the core pieces of AMQP-style messaging back to their
essences.

### More detail

TBD.

 - what recursive means in this context
 - doing things this way gives you shovels (relays) for free
 - and effortless interop with legacy messaging networks (including UDP, SMTP, IMAP, HTTP etc)
 - and effortless federation
 - and a big step closer to a sensible semantics for transactions

 - relays (including active *client* connections!) are just nodes in
   the network, addressable like any other - so `(post! somerelay
   (post! someremotenode ...))` and so on is the way to cause things
   to happen remotely.

## Compiling the server

The server is written in [OCaml](http://caml.inria.fr/). To build and
run the server, you will need:

 - [OCaml itself](http://caml.inria.fr/download.en.html), version 3.12 or newer
 - [OCaml Findlib](http://projects.camlcity.org/projects/findlib.html); I have used 1.2.8 and 1.3.1, but older versions may well work
 - [libev](http://software.schmorp.de/pkg/libev.html) installed somewhere that Findlib can find it
 - [python](http://www.python.org/) to generate parts of the protocol codecs

Make sure you have `ocamlopt`, `ocamlbuild`, `ocamlfind` etc. on your
path. Then, in the `server` subdirectory, run `make`. It should first
compile [Lwt](http://ocsigen.org/lwt/), which is included as a
third-party library, and then should proceed to compiling the server
itself.

If `ocamlfind` can't find `libev`, try setting (and exporting) the
environment variables `C_INCLUDE_PATH` and `LIBRARY_PATH` to point to
the include and lib directories containing `libev`'s files.

To run the server, simply run `./server/hop_server.native`, or just
`make run` from within the `server` directory.

## Working with the management and monitoring webpages

If you want to edit and/or recompile the server's built-in webpages,
you will need to have installed

 - [xsltproc](http://xmlsoft.org/xslt/xsltproc2.html) to make the webpages from the templates
 - [recess](http://twitter.github.com/recess/) to compile the LESS into CSS

## Compiling the Java client library

You will need a recent JDK, and Ant v1.6 or newer. Change to the
`java` subdirectory, and run `ant`. You will end up with a file
`build/lib/hop.jar`, which contains the client library and some test
programs.

## Run it

Open three terminals. Run the server in one of them. You should see
output like the following:

    hop ALPHA, Copyright (C) 2012 Tony Garnock-Jones.
    This program comes with ABSOLUTELY NO WARRANTY. This is free software,
    and you are welcome to redistribute it under certain conditions.
    See the GNU General Public License (version 3 or later) for details.
    info: ("Node bound" "factory" "factory")
    info: ("Registered node class" "queue")
    info: ("Registered node class" "fanout")
    info: ("Registered node class" "direct")
    info: ("Node bound" "meta" "direct")
    info: ("Node create ok" "direct" ("meta") "" "" "meta")
    info: ("Node bound" "amq.direct" "direct")
    info: ("Node create ok" "direct" ("amq.direct") "" "" "amq.direct")
    info: ("Node bound" "amq.fanout" "fanout")
    info: ("Node create ok" "fanout" ("amq.fanout") "" "" "amq.fanout")
    info: ("Accepting connections" "AMQP" "5672")
    info: ("Accepting connections" "HTTP" "5678")
    info: ("Accepting connections" "Hop" "5671")
    info: ("Waiting for milestone" "AMQP ready")
    info: ("Achieved milestone" "AMQP ready")
    info: ("Waiting for milestone" "HTTP ready")
    info: ("Achieved milestone" "HTTP ready")
    info: ("Waiting for milestone" "Hop ready")
    info: ("Achieved milestone" "Hop ready")
    info: ("Achieved milestone" "Server initialized")

In the second terminal, run the consuming half of the Java test
program pair:

    java -cp hop.jar hop.Test1 localhost

In the third, run the producing half:

    java -cp hop.jar hop.Test3 localhost

## Wire protocol

Obviously the wire protocol itself here is the simplest thing that
could possibly work, and you'd never use anything like this
inefficient in a real system. That said, this is what's there right
now:

### Message transfer

`(post <routing-key> <message> <subscription-token>)` - Instructs the
receiving node to route (or process) the given `message` according to
the given `routing-key`. Different kinds of nodes will do different
things here, and in particular, will interpret the routing key
differently. Queues, for example, will ignore the routing key and will
deliver the message to only one of their active subscribers, whereas
exchanges will generally match the routing key against their active
subscriptions and will deliver the message on to all matches.

### Subscription management

`(subscribe <routing-key-filter> <target-node> <target-routing-key>
<reply-node> <reply-routing-key>)` - Instructs the receiving node to
create a new subscription. The new subscription will only route
messages matching the `routing-key-filter`, which is interpreted on a
per-node-type basis as above for `routing-key`. Matching messages will
be sent to `target-node` using `post!`, with a routing key of
`target-routing-key`. The `reply-node` parameter, if nonempty,
instructs the receiving node to send confirmation of subscription
(along with a token that can be used with `unsubscribe` below) to the
given address and routing key. If `reply-node` is empty, no
confirmation of subscription is sent.

`(unsubscribe <token>)` - Instructs the receiving node to delete a
previously established subscription. The `token` comes from the
`subscribe-ok` message sent to `reply-node` after a successful
`subscribe` operation.

### Object management

`(create <class-name> <argument> <reply-node> <reply-routing-key>)` -
Instructs the receiving object factory node to construct a new
instance of the given `class-name`, with the given `argument` supplied
to the constructor. The `reply-node` and `reply-routing-key` are used
to send confirmation of completion to some waiting node.

## Copyright and licensing

Hop is Copyright 2010, 2011, 2012 Tony Garnock-Jones
<tonygarnockjones@gmail.com>.

Hop is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation, either version 3 of the License, or (at your
option) any later version.

Hop is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
for more details.

You should have received a copy of the GNU General Public License
along with Hop.  If not, see <http://www.gnu.org/licenses/>.
