036916de7f | ||
---|---|---|
racketmq | ||
.gitignore | ||
COPYING | ||
Makefile | ||
README.md | ||
gpl.txt | ||
info.rkt | ||
lgpl.txt | ||
nginx.conf |
README.md
RacketMQ: An implementation of W3C WebSub
This is an implementation of a W3C WebSub Hub in Racket, using the actor-style research language Syndicate.
What is WebSub?
On the 20th of October 2016, the W3C released a First Public Working Draft of (what was called at the time) PubSub, later renamed to WebSub.
See the specification of the W3C WebSub protocol at
https://www.w3.org/TR/pubsub/ (and track its development at
https://github.com/w3c/pubsub).
N.B.: These URLs will eventually have websub
in place of pubsub
.
Quick Start
- Install Racket from http://download.racket-lang.org/
- Install RacketMQ by running
raco pkg install --auto racketmq
racketmq --canonical-host localhost 7827
To install from git, replace the raco pkg install ...
step above
with an invocation of make link
from the top directory of your git
checkout.
Features
-
Offers both local topics, topics whose canonical hub is this hub, and remote topics, topics whose canonical hub is some other ("upstream") hub
-
Support for polling and push-notification for remote topics, with configurable poll interval
-
Uses HTTP
Link
headers when retrieving a topic to determine canonical hub and topic URLs; does not extractlink
elements from any kind of XML or HTML document, nor does it implement.host-meta
discovery -
Support for
hub.secret
andhub.lease_seconds
protocol parameters
Configuration
RacketMQ has only one required configuration variable: you must tell the hub its primary ("canonical") host name and port number. These are used to build URLs for clients of the Hub to use.
When the RacketMQ startup script is given a "-f
filename" option,
it loads configuration data from the named file. The option can be
supplied more than once; all named files are imported.
For a fully-commented example configuration file, see
racketmq/defaults.rktd
.
Within each file, each configuration entry should be a list (see Racket syntax) with a symbol (the "key") as its first item followed by zero or more items.
Each configuration file is automatically reread by the server when it
is changed: if you need to make changes, consider doing so atomically
by producing an updated configuration file and using
rename(2)
/mv(1)
to activate it.
Required configuration data
(canonical-host "localhost" 7827)
Exactly one "canonical-host" key, containing two values, a string hostname, and a number TCP port. This causes an HTTP server to be spun up on the named port.
Since this is the only mandatory configuration item, RacketMQ can run
without any configuration file at all if the server is started with
the --canonical-host
command-line argument:
racketmq --canonical-host localhost 7827
Optional configuration data
(accepted-host "localhost" 7827)
(accepted-host "localhost" 80)
(accepted-host "www.example.com" 7827)
;; etc.
If you want your server to appear under several aliases, add them
here. HTTP servers will be spun up for all mentioned port numbers,
and within those servers, Host
headers matching the given host
names will be accepted.
Fine tuning
You will seldom want to alter these settings.
(max-upstream-redirects 5)
When performing discovery / upstream content retrieval, the hub will follow this many redirects before deciding it has had enough.
(default-lease 86400) ;; 86400 seconds = one day
(max-lease 604800) ;; 604800 seconds = one week
If a subscription request arrives with no specified
hub.lease_seconds
, then default-lease
is used. If a requested
lease duration exceeds max-lease
, then max-lease
is used
instead.
(min-poll-interval 60) ;; seconds
(default-poll-interval "none") ;; seconds, or "none"
Upstream topics will be polled from time to time, according to the
settings of each local subscription to the topic. Subscriptions may
supply hub.poll_interval_seconds
as either a number or the string
"none". If no hub.poll_interval_seconds
is supplied in a
subscription, default-poll-interval
is used. If all subscriptions
to an upstream topic have "none" as their poll interval, no polling
will occur; otherwise, polling will occur at the fastest requested
rate, but never more frequently than every min-poll-interval
seconds.
(max-dead-letters 10)
(max-delivery-retries 10)
(initial-retry-delay 5.0) ;; seconds
(retry-delay-multiplier 1.618)
(max-retry-delay 30) ;; seconds
Subscriptions last until explicitly terminated by an unsubscription request, implicitly terminated by lease expiry, or implicitly terminated by sustained delivery failure.
When the hub sends a content distribution request (see the WebSub spec) to a subscription's callback, if a success response is returned, the delivery is considered successful.
Otherwise, the hub begins an exponential backoff process, with an
initial delay of initial-retry-delay
seconds, increasing by a
factor of retry-delay-multiplier
(subject to a cap of
max-retry-delay
seconds) with each subsequent attempt until
max-delivery-retries
attempts have been made. At that point, if
all attempts to deliver the particular content distribution request
have failed, the request is considered a "dead letter" and is
effectively discarded. Once a request has either succeeded or
become a dead letter, the hub continues with any further pending
content distribution requests for the subscription.
If more than max-dead-letters
dead letters pile up for a
subscription, the subscription is considered too damaged to
continue to exist, and is terminated.
Hub URL layout
-
/hub
— Local subscription management; main Hub URL.This is the main URL for creating and deleting subscriptions to (local or remote) topics.
- method
POST
: create or delete a subscription, following the specification. Supplyhub.mode
,hub.topic
,hub.callback
and other relevant parameters to manage subscriptions.
- method
-
/topic/
topic — Local topic endpoint.A local topic is a topic managed by this hub. Publishers
POST
their content to the local topic endpoint, and subscribers are notified of the change. Local topics may be managed explicitly or implicitly; any subscription to a local topic will automatically cause it to be created, even if it has not been previously explicitlyPUT
into existence.- method
PUT
: create a local topic explicitly - method
DELETE
: delete an explicitly-created local topic - method
HEAD
: get headers associated with the most recent topic value - method
GET
: get the most recent topic value - method
POST
: update the topic value with the post body
- method
-
/sub/
sub-id — Upstream subscription endpoint.When a subscription to a remote topic is created, if the remote topic has an advertised hub, this hub subscribes to the remote hub, and content distribution requests are
POST
ed to a fresh upstream subscription endpoint URL.- method
GET
: for verification-of-intent requests from upstream. - method
POST
: for content distribution requests from upstream.
- method
-
/
path/to/file/in/htdocs — Static resource.The
racketmq/htdocs
subdirectory contains static resources to be served by the hub.- method
GET
: retrieve a static resource.
- method
Conformance
At the time of writing, no official list of conformance criteria exists; however, there is a draft list of Candidate Recommendation implementation criteria at https://github.com/w3c/pubsub/issues/56.
Bug Reports
Please report issues using this project's Github issues page, https://github.com/tonyg/racketmq/issues.
License
Copyright © 2016 Tony Garnock-Jones tonyg@leastfixedpoint.com
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with this program (see the files "lgpl.txt" and "gpl.txt"). If not, see http://www.gnu.org/licenses/.