197 lines
6.8 KiB
Markdown
197 lines
6.8 KiB
Markdown
|
# The Hop Pattern
|
||
|
|
||
|
Hop is a combination of
|
||
|
|
||
|
- a syntax for encoding data for transport
|
||
|
- a subscription and messaging protocol
|
||
|
- a design for recursive networks
|
||
|
|
||
|
In this document, I will sketch the third aspect of the system: the
|
||
|
design for recursive networking.
|
||
|
|
||
|
## Core idea: Abstract Networks
|
||
|
|
||
|
Every virtual machine, TCP connection, overlay network, and other
|
||
|
object in the system can be viewed abstractly as a communications
|
||
|
network. This applies just as well to individual TCP connections and
|
||
|
individual Java objects as it does to AMQP-style exchanges, brokers,
|
||
|
and queues.
|
||
|
|
||
|
This might not seem like an advantage, but in fact looking at things
|
||
|
this way permits simple, regular composition of networks (and virtual
|
||
|
machines, etc.).
|
||
|
|
||
|
### Abstract Networks
|
||
|
|
||
|
The idea of an abstract network that we will be using is this:
|
||
|
|
||
|
A network is a *namespace* within which exist *nodes* which send
|
||
|
*messages* that are addressed to other *names* in the network. Names
|
||
|
are mapped back to nodes via a *directory* of some kind. Joining such
|
||
|
a network is called *enrollment*: it is the process of, first,
|
||
|
authenticating a node to the network, and second, *binding* of zero or
|
||
|
more names in the namespace to the new node. These networks also have
|
||
|
a *routing semantic*: when a message is sent to some name in the
|
||
|
network, the network will have a characteristic way of using its
|
||
|
directory to decide which, if any, nodes in the network should receive
|
||
|
the message.
|
||
|
|
||
|
### A TCP connection, viewed as an abstract network
|
||
|
|
||
|
A single TCP connection can be seen as a network.
|
||
|
|
||
|
- the set of possible names is exactly `{client, server}`.
|
||
|
- joining the network and binding to names happens automatically as
|
||
|
part of the creation of the connection.
|
||
|
- messages sent by one party are implicitly delivered to the other
|
||
|
party.
|
||
|
|
||
|
### A Java virtual machine instance, viewed as an abstract network
|
||
|
|
||
|
A Java virtual machine and all its objects can be seen as a network.
|
||
|
|
||
|
- the set of possible names is the set of valid (live) pointers to
|
||
|
objects.
|
||
|
- joining the network and binding to names happens automatically each
|
||
|
time a new object is created.
|
||
|
- object names are (almost) capabilities.
|
||
|
- method calls, returns, and exceptions are the messages sent in the
|
||
|
system: calls are sent to object names, returns to continuation
|
||
|
names (transient and implicitly specified), and exceptions to
|
||
|
handler names (again implicitly specified).
|
||
|
- there is a built-in (and unavoidable) notion of *conversation
|
||
|
pattern* included in the semantic of this kind of network: each
|
||
|
method call message results in either a return message or an
|
||
|
exception message.
|
||
|
|
||
|
### The world-wide UDP/IP network, viewed as an abstract network
|
||
|
|
||
|
The entire collection of UDP/IP endpoints can be seen as an abstract
|
||
|
network.
|
||
|
|
||
|
- the names are the `{ip, port}` pairs.
|
||
|
- joining the network and binding to a name happens in part via the
|
||
|
operation of DHCP (giving an IP address) and in part via, for
|
||
|
example, BSD sockets' `bind(2)` system call.
|
||
|
- messages sent in the system are just UDP packets.
|
||
|
|
||
|
### An AMQP broker, viewed as an abstract network
|
||
|
|
||
|
AMQP 0-9-1 brokers can be seen as abstract networks.
|
||
|
|
||
|
- the nodes are the exchanges, queues, consumers and active channels
|
||
|
in the broker.
|
||
|
- some nodes, namely queues and exchanges, join the network via the
|
||
|
operation of a broker-internal *factory* node.
|
||
|
- other nodes, namely channels and consumers, join the network via
|
||
|
connections from the outside world.
|
||
|
- name binding for nodes happens at the time they're created.
|
||
|
|
||
|
### An AMQP "direct" exchange, viewed as an abstract network
|
||
|
|
||
|
Exchanges within AMQP 0-9-1 brokers can be seen as abstract
|
||
|
networks. For example, consider "direct"-style exchanges, though
|
||
|
similar observations can be made of the other kinds of exchange
|
||
|
defined by the AMQP specification:
|
||
|
|
||
|
- the names are the routing-keys to which queues have been bound.
|
||
|
- joining the network and binding to names is done via queue binding:
|
||
|
queues are the only nodes in AMQP that can join an exchange's
|
||
|
network.
|
||
|
- messages delivered to an exchange are routed using the exchange's
|
||
|
internal routing table (its directory) to zero or more bound
|
||
|
queues.
|
||
|
|
||
|
## Bridging between abstract networks
|
||
|
|
||
|
So far, I've mentioned two kinds of network: those where there is no
|
||
|
explicit notion of a bridge to the outside world, such as individual
|
||
|
TCP connections, Java virtual machine instances, and the worldwide
|
||
|
UDP/IP network, and those where some explicit idea of connecting to
|
||
|
other systems is present, such as AMQP brokers and exchanges.
|
||
|
|
||
|
Considering UDP/IP for a moment, it's clear that the "layer 2"
|
||
|
protocols underpinning UDP/IP act in some way as connections to other
|
||
|
fragments of the worldwide network. Similarly, each TCP/IP connection
|
||
|
has a kind of "border router" at each end embodied in the BSD sockets
|
||
|
API that routes packets to and from the kernel's TCP stack. Finally,
|
||
|
Java VM instances have a plethora of options for communicating with
|
||
|
the outside world and permitting communications from the outside world
|
||
|
to cause methods to be invoked on Java objects—that is, from the
|
||
|
abstract point of view, to cause messages to be sent within the VM's
|
||
|
internal network.
|
||
|
|
||
|
((TODO: distinguish carefully between interior routers and border
|
||
|
routers: the former glue together subnets of UDP/IP into a single
|
||
|
worldwide namespace, for example, where the latter permit access to
|
||
|
other namespaces, or even kinds of namespace, from a given
|
||
|
network. The former implement a distributed directory and routing
|
||
|
system, the latter embed entire networks as single nodes in a host
|
||
|
network.))
|
||
|
|
||
|
It seems clear that every kind of complete computing system has three aspects:
|
||
|
|
||
|
- a computation facility (which is how it does its job, whatever that may be)
|
||
|
- an internal communications facility (its network aspect)
|
||
|
- an external communications facility (routing of data to other systems/networks)
|
||
|
|
||
|
The TCP connections that AMQP brokers accept, the bindings between
|
||
|
AMQP exchanges and queues, the "consumer" relationships between AMQP
|
||
|
queues and channels,
|
||
|
|
||
|
### Abstract relays
|
||
|
|
||
|
Abstract relays *embed* a remote network as a single node within a
|
||
|
local network. Messages sent via local names to relay instances are
|
||
|
transported uninterpreted across some underlying transport to the
|
||
|
other network, where a corresponding relay instance interprets the
|
||
|
messages being sent. A symmetric relay setup may also exist, which
|
||
|
gives a *mutual embedding* of the two networks.
|
||
|
|
||
|
## Core protocol
|
||
|
|
||
|
`post`
|
||
|
|
||
|
`subscribe`
|
||
|
|
||
|
`unsubscribe`
|
||
|
|
||
|
## Refinements
|
||
|
|
||
|
Acknowledgement
|
||
|
|
||
|
Lifetime-coupling
|
||
|
|
||
|
## Frequently useful node classes
|
||
|
|
||
|
Factory
|
||
|
|
||
|
Relay (seen in many various guises)
|
||
|
|
||
|
Exchange
|
||
|
|
||
|
Queue/Buffer
|
||
|
|
||
|
Consumer
|
||
|
|
||
|
Broker (usually explicit only as part of a mutual embedding of a relay
|
||
|
into a network)
|
||
|
|
||
|
## Elements
|
||
|
|
||
|
An implementation of the Hop pattern will include
|
||
|
|
||
|
Codec
|
||
|
|
||
|
relay
|
||
|
|
||
|
classes
|
||
|
|
||
|
nodes
|
||
|
|
||
|
namespace and a dispatcher
|
||
|
|
||
|
factory
|
||
|
|
||
|
a notion of subscription, same as the notion of binding
|