197 lines
7.9 KiB
Markdown
197 lines
7.9 KiB
Markdown
# Recursive Message Broker
|
|
|
|
This is a *sketch* of a recursive messaging protocol, broker, and
|
|
client libraries, inspired by AMQP 0-91,
|
|
[PubSubHubBub](http://code.google.com/p/pubsubhubbub/),
|
|
[STOMP](http://stomp.github.com/) and
|
|
[reversehttp](http://reversehttp.net/). It's quite different to AMQP
|
|
1.0 (but it may be instructive to compare the two approaches).
|
|
|
|
Currently, the project includes
|
|
|
|
- a server (written in OCaml), with adapters for
|
|
- Hop's own protocol,
|
|
- a subset of AMQP 0-9-1, and
|
|
- XHR streaming of messages to and from the broker.
|
|
- a web-based console for the server
|
|
- an OSX GUI for the server
|
|
- messaging for the web
|
|
- client libraries for various languages (Java, Racket, Javascript)
|
|
|
|
## A sketch?
|
|
|
|
Honestly, not meant to be production software... yet.
|
|
|
|
## Background
|
|
|
|
Messaging à la AMQP 0-91 can be broken down into a few core pieces:
|
|
|
|
- transmission and receipt of messages (publishes, deliveries, and gets)
|
|
|
|
- subscription management (subsuming enrollment, bindings, consumers and relays)
|
|
|
|
- directory (naming of resources in the broker)
|
|
|
|
- object management (creation and destruction of remote resources)
|
|
|
|
AMQP itself, being a first mover in its space, isn't as orthogonal as
|
|
it could be. It can be greatly simplified without losing anything of
|
|
value. This experiment is intended to demonstrate one possible way of
|
|
paring each of the core pieces of AMQP-style messaging back to their
|
|
essences.
|
|
|
|
### More detail
|
|
|
|
TBD.
|
|
|
|
- what recursive means in this context
|
|
- doing things this way gives you shovels (relays) for free
|
|
- and effortless interop with legacy messaging networks (including UDP, SMTP, IMAP, HTTP etc)
|
|
- and effortless federation
|
|
- and a big step closer to a sensible semantics for transactions
|
|
|
|
- relays (including active *client* connections!) are just nodes in
|
|
the network, addressable like any other - so `(post! somerelay
|
|
(post! someremotenode ...))` and so on is the way to cause things
|
|
to happen remotely.
|
|
|
|
## Compiling the server
|
|
|
|
The server is written in [OCaml](http://caml.inria.fr/). To build and
|
|
run the server, you will need:
|
|
|
|
- [OCaml itself](http://caml.inria.fr/download.en.html), version 3.12 or newer
|
|
- [OCaml Findlib](http://projects.camlcity.org/projects/findlib.html); I have used 1.2.8 and 1.3.1, but older versions may well work
|
|
- [libev](http://software.schmorp.de/pkg/libev.html) installed somewhere that Findlib can find it
|
|
- [python](http://www.python.org/) to generate parts of the protocol codecs
|
|
|
|
Make sure you have `ocamlopt`, `ocamlbuild`, `ocamlfind` etc. on your
|
|
path. Then, in the `server` subdirectory, run `make`. It should first
|
|
compile [Lwt](http://ocsigen.org/lwt/), which is included as a
|
|
third-party library, and then should proceed to compiling the server
|
|
itself.
|
|
|
|
If `ocamlfind` can't find `libev`, try setting (and exporting) the
|
|
environment variables `C_INCLUDE_PATH` and `LIBRARY_PATH` to point to
|
|
the include and lib directories containing `libev`'s files.
|
|
|
|
To run the server, simply run `./server/hop_server.native`, or just
|
|
`make run` from within the `server` directory.
|
|
|
|
## Working with the management and monitoring webpages
|
|
|
|
If you want to edit and/or recompile the server's built-in webpages,
|
|
you will need to have installed
|
|
|
|
- [xsltproc](http://xmlsoft.org/xslt/xsltproc2.html) to make the webpages from the templates
|
|
- [recess](http://twitter.github.com/recess/) to compile the LESS into CSS
|
|
|
|
## Compiling the Java client library
|
|
|
|
You will need a recent JDK, and Ant v1.6 or newer. Change to the
|
|
`java` subdirectory, and run `ant`. You will end up with a file
|
|
`build/lib/hop.jar`, which contains the client library and some test
|
|
programs.
|
|
|
|
## Run it
|
|
|
|
Open three terminals. Run the server in one of them. You should see
|
|
output like the following:
|
|
|
|
hop ALPHA, Copyright (C) 2012 Tony Garnock-Jones.
|
|
This program comes with ABSOLUTELY NO WARRANTY. This is free software,
|
|
and you are welcome to redistribute it under certain conditions.
|
|
See the GNU General Public License (version 3 or later) for details.
|
|
info: ("Node bound" "factory" "factory")
|
|
info: ("Registered node class" "queue")
|
|
info: ("Registered node class" "fanout")
|
|
info: ("Registered node class" "direct")
|
|
info: ("Node bound" "meta" "direct")
|
|
info: ("Node create ok" "direct" ("meta") "" "" "meta")
|
|
info: ("Node bound" "amq.direct" "direct")
|
|
info: ("Node create ok" "direct" ("amq.direct") "" "" "amq.direct")
|
|
info: ("Node bound" "amq.fanout" "fanout")
|
|
info: ("Node create ok" "fanout" ("amq.fanout") "" "" "amq.fanout")
|
|
info: ("Accepting connections" "AMQP" "5672")
|
|
info: ("Accepting connections" "HTTP" "5678")
|
|
info: ("Accepting connections" "Hop" "5671")
|
|
info: ("Waiting for milestone" "AMQP ready")
|
|
info: ("Achieved milestone" "AMQP ready")
|
|
info: ("Waiting for milestone" "HTTP ready")
|
|
info: ("Achieved milestone" "HTTP ready")
|
|
info: ("Waiting for milestone" "Hop ready")
|
|
info: ("Achieved milestone" "Hop ready")
|
|
info: ("Achieved milestone" "Server initialized")
|
|
|
|
In the second terminal, run the consuming half of the Java test
|
|
program pair:
|
|
|
|
java -cp hop.jar hop.Test1 localhost
|
|
|
|
In the third, run the producing half:
|
|
|
|
java -cp hop.jar hop.Test3 localhost
|
|
|
|
## Wire protocol
|
|
|
|
Obviously the wire protocol itself here is the simplest thing that
|
|
could possibly work, and you'd never use anything like this
|
|
inefficient in a real system. That said, this is what's there right
|
|
now:
|
|
|
|
### Message transfer
|
|
|
|
`(post <routing-key> <message> <subscription-token>)` - Instructs the
|
|
receiving node to route (or process) the given `message` according to
|
|
the given `routing-key`. Different kinds of nodes will do different
|
|
things here, and in particular, will interpret the routing key
|
|
differently. Queues, for example, will ignore the routing key and will
|
|
deliver the message to only one of their active subscribers, whereas
|
|
exchanges will generally match the routing key against their active
|
|
subscriptions and will deliver the message on to all matches.
|
|
|
|
### Subscription management
|
|
|
|
`(subscribe <routing-key-filter> <target-node> <target-routing-key>
|
|
<reply-node> <reply-routing-key>)` - Instructs the receiving node to
|
|
create a new subscription. The new subscription will only route
|
|
messages matching the `routing-key-filter`, which is interpreted on a
|
|
per-node-type basis as above for `routing-key`. Matching messages will
|
|
be sent to `target-node` using `post!`, with a routing key of
|
|
`target-routing-key`. The `reply-node` parameter, if nonempty,
|
|
instructs the receiving node to send confirmation of subscription
|
|
(along with a token that can be used with `unsubscribe` below) to the
|
|
given address and routing key. If `reply-node` is empty, no
|
|
confirmation of subscription is sent.
|
|
|
|
`(unsubscribe <token>)` - Instructs the receiving node to delete a
|
|
previously established subscription. The `token` comes from the
|
|
`subscribe-ok` message sent to `reply-node` after a successful
|
|
`subscribe` operation.
|
|
|
|
### Object management
|
|
|
|
`(create <class-name> <argument> <reply-node> <reply-routing-key>)` -
|
|
Instructs the receiving object factory node to construct a new
|
|
instance of the given `class-name`, with the given `argument` supplied
|
|
to the constructor. The `reply-node` and `reply-routing-key` are used
|
|
to send confirmation of completion to some waiting node.
|
|
|
|
## Copyright and licensing
|
|
|
|
Hop is Copyright 2010, 2011, 2012 Tony Garnock-Jones
|
|
<tonygarnockjones@gmail.com>.
|
|
|
|
Hop is free software: you can redistribute it and/or modify it under
|
|
the terms of the GNU General Public License as published by the Free
|
|
Software Foundation, either version 3 of the License, or (at your
|
|
option) any later version.
|
|
|
|
Hop is distributed in the hope that it will be useful, but WITHOUT
|
|
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
|
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
|
for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with Hop. If not, see <http://www.gnu.org/licenses/>.
|