diff --git a/doc/hop.md b/doc/hop.md new file mode 100644 index 0000000..149a213 --- /dev/null +++ b/doc/hop.md @@ -0,0 +1,196 @@ +# The Hop Pattern + +Hop is a combination of + + - a syntax for encoding data for transport + - a subscription and messaging protocol + - a design for recursive networks + +In this document, I will sketch the third aspect of the system: the +design for recursive networking. + +## Core idea: Abstract Networks + +Every virtual machine, TCP connection, overlay network, and other +object in the system can be viewed abstractly as a communications +network. This applies just as well to individual TCP connections and +individual Java objects as it does to AMQP-style exchanges, brokers, +and queues. + +This might not seem like an advantage, but in fact looking at things +this way permits simple, regular composition of networks (and virtual +machines, etc.). + +### Abstract Networks + +The idea of an abstract network that we will be using is this: + +A network is a *namespace* within which exist *nodes* which send +*messages* that are addressed to other *names* in the network. Names +are mapped back to nodes via a *directory* of some kind. Joining such +a network is called *enrollment*: it is the process of, first, +authenticating a node to the network, and second, *binding* of zero or +more names in the namespace to the new node. These networks also have +a *routing semantic*: when a message is sent to some name in the +network, the network will have a characteristic way of using its +directory to decide which, if any, nodes in the network should receive +the message. + +### A TCP connection, viewed as an abstract network + +A single TCP connection can be seen as a network. + + - the set of possible names is exactly `{client, server}`. + - joining the network and binding to names happens automatically as + part of the creation of the connection. + - messages sent by one party are implicitly delivered to the other + party. + +### A Java virtual machine instance, viewed as an abstract network + +A Java virtual machine and all its objects can be seen as a network. + + - the set of possible names is the set of valid (live) pointers to + objects. + - joining the network and binding to names happens automatically each + time a new object is created. + - object names are (almost) capabilities. + - method calls, returns, and exceptions are the messages sent in the + system: calls are sent to object names, returns to continuation + names (transient and implicitly specified), and exceptions to + handler names (again implicitly specified). + - there is a built-in (and unavoidable) notion of *conversation + pattern* included in the semantic of this kind of network: each + method call message results in either a return message or an + exception message. + +### The world-wide UDP/IP network, viewed as an abstract network + +The entire collection of UDP/IP endpoints can be seen as an abstract +network. + + - the names are the `{ip, port}` pairs. + - joining the network and binding to a name happens in part via the + operation of DHCP (giving an IP address) and in part via, for + example, BSD sockets' `bind(2)` system call. + - messages sent in the system are just UDP packets. + +### An AMQP broker, viewed as an abstract network + +AMQP 0-9-1 brokers can be seen as abstract networks. + + - the nodes are the exchanges, queues, consumers and active channels + in the broker. + - some nodes, namely queues and exchanges, join the network via the + operation of a broker-internal *factory* node. + - other nodes, namely channels and consumers, join the network via + connections from the outside world. + - name binding for nodes happens at the time they're created. + +### An AMQP "direct" exchange, viewed as an abstract network + +Exchanges within AMQP 0-9-1 brokers can be seen as abstract +networks. For example, consider "direct"-style exchanges, though +similar observations can be made of the other kinds of exchange +defined by the AMQP specification: + + - the names are the routing-keys to which queues have been bound. + - joining the network and binding to names is done via queue binding: + queues are the only nodes in AMQP that can join an exchange's + network. + - messages delivered to an exchange are routed using the exchange's + internal routing table (its directory) to zero or more bound + queues. + +## Bridging between abstract networks + +So far, I've mentioned two kinds of network: those where there is no +explicit notion of a bridge to the outside world, such as individual +TCP connections, Java virtual machine instances, and the worldwide +UDP/IP network, and those where some explicit idea of connecting to +other systems is present, such as AMQP brokers and exchanges. + +Considering UDP/IP for a moment, it's clear that the "layer 2" +protocols underpinning UDP/IP act in some way as connections to other +fragments of the worldwide network. Similarly, each TCP/IP connection +has a kind of "border router" at each end embodied in the BSD sockets +API that routes packets to and from the kernel's TCP stack. Finally, +Java VM instances have a plethora of options for communicating with +the outside world and permitting communications from the outside world +to cause methods to be invoked on Java objects—that is, from the +abstract point of view, to cause messages to be sent within the VM's +internal network. + +((TODO: distinguish carefully between interior routers and border +routers: the former glue together subnets of UDP/IP into a single +worldwide namespace, for example, where the latter permit access to +other namespaces, or even kinds of namespace, from a given +network. The former implement a distributed directory and routing +system, the latter embed entire networks as single nodes in a host +network.)) + +It seems clear that every kind of complete computing system has three aspects: + + - a computation facility (which is how it does its job, whatever that may be) + - an internal communications facility (its network aspect) + - an external communications facility (routing of data to other systems/networks) + +The TCP connections that AMQP brokers accept, the bindings between +AMQP exchanges and queues, the "consumer" relationships between AMQP +queues and channels, + +### Abstract relays + +Abstract relays *embed* a remote network as a single node within a +local network. Messages sent via local names to relay instances are +transported uninterpreted across some underlying transport to the +other network, where a corresponding relay instance interprets the +messages being sent. A symmetric relay setup may also exist, which +gives a *mutual embedding* of the two networks. + +## Core protocol + +`post` + +`subscribe` + +`unsubscribe` + +## Refinements + +Acknowledgement + +Lifetime-coupling + +## Frequently useful node classes + +Factory + +Relay (seen in many various guises) + +Exchange + +Queue/Buffer + +Consumer + +Broker (usually explicit only as part of a mutual embedding of a relay +into a network) + +## Elements + +An implementation of the Hop pattern will include + +Codec + +relay + +classes + +nodes + +namespace and a dispatcher + +factory + +a notion of subscription, same as the notion of binding