syndicate-lang
/
racketmq-2017
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
racketmq-2017/racketmq/hub/remote-topic.rkt

239 lines
10 KiB
Racket
Raw Normal View History

Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
#lang syndicate/actor
(require racket/dict)
(require racket/set)
(require file/sha1)
(require/activate syndicate/drivers/timestate)
(require/activate syndicate/drivers/web)
(require/activate "../config.rkt")
(require "../private/util.rkt")
(require "../protocol.rkt")
Fixes and improvements toward proper upstream subscriptions
2016-11-21 21:04:07 +00:00
(module+ test (require rackunit))
Minor refactoring
2016-11-21 04:05:01 +00:00
(define (set-minimum xs)
(for/fold [(i #f)] [(candidate (in-set xs))]
(cond [(not candidate) i]
[(not i) candidate]
[(< candidate i) candidate]
[else i])))
Fixes and improvements toward proper upstream subscriptions
2016-11-21 21:04:07 +00:00
(define (shrink-lease seconds)
(if (< seconds 100)
(* seconds 9/10)
(- seconds 10)))
(module+ test
(check-equal? (shrink-lease 1000) 990)
(check-equal? (shrink-lease 101) 91)
(check-equal? (shrink-lease 100) 90)
(check-equal? (shrink-lease 90) 81)
(check-equal? (shrink-lease 50) 45))
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
;; An UpstreamLink represents demand for a subscription to a remote
;; hub. Notifications received should be forwarded locally using
;; `requested-topic` as the first field of a `notification` message.
(struct upstream-link (requested-topic ;; Topic -- the topic requested by *our* subscriber
upstream-hub ;; URLString -- upstream hub
upstream-topic ;; Topic -- upstream topic (from discovery)
) #:prefab) ;; ASSERTION
Reinterpret canonical-local-host -> canonical-baseurl. In order to support HTTPS nginx proxying, we need to maintain a "fictional" baseurl which is not directly connected to this server's listen ports. This has caused a change to configuration: now, a baseurl is specified along with one or more listen specifications.
2016-11-21 22:35:39 +00:00
(actor #:name 'remote-topic-manager
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(during/actor (remote-topic-demand $requested-topic)
#:name (list 'remote-topic requested-topic)
(remote-topic-main requested-topic)))
Reinterpret canonical-local-host -> canonical-baseurl. In order to support HTTPS nginx proxying, we need to maintain a "fictional" baseurl which is not directly connected to this server's listen ports. This has caused a change to configuration: now, a baseurl is specified along with one or more listen specifications.
2016-11-21 22:35:39 +00:00
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(actor #:name 'upstream-link-manager
(during/actor (upstream-link $requested-topic $upstream-hub $upstream-topic)
#:name (list 'upstream-link requested-topic upstream-hub upstream-topic)
(upstream-link-main requested-topic upstream-hub upstream-topic)))
(define (remote-topic-main requested-topic)
;; Does discovery and occasionally polls.
(on-start (log-info "Remote topic ~s starting." requested-topic))
(on-stop (log-info "Remote topic ~s terminating." requested-topic))
Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
(field [current-content-hash #f]
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
[current-content-type #f])
(field [current-upstream-hub #f]
[current-upstream-topic requested-topic])
Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
(field [last-upstream-check 0]
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
[poll-interval-seconds #f])
Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
(define/query-config min-poll-interval 60)
Fixes and improvements toward proper upstream subscriptions
2016-11-21 21:04:07 +00:00
(define/query-config max-upstream-redirects 5)
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(define/query-set poll-intervals (topic-demand requested-topic $i) i)
Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
(begin/dataflow
Minor refactoring
2016-11-21 04:05:01 +00:00
(define candidate (set-minimum (poll-intervals)))
Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
(poll-interval-seconds (and candidate (max candidate (min-poll-interval)))))
(begin/dataflow
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(log-info "Poll interval for ~s is now ~a"
requested-topic
Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
(match (poll-interval-seconds)
[#f "disabled"]
[n (format "~a seconds" n)])))
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(field [poll-busy? #f]) ;; Ensure we only run one poll at once
(define (poll-upstream!)
(when (not (poll-busy?))
(define poll-upstream-topic (current-upstream-topic))
(poll-busy? #t)
(log-info "Checking upstream ~a" poll-upstream-topic)
(define-values (resp response-body)
(web-request! 'get poll-upstream-topic #:redirect-budget (max-upstream-redirects)))
(last-upstream-check (current-inexact-milliseconds))
(match (web-response-header-code-type resp)
['successful
(define parsed-link-headers (parse-link-headers (web-response-header-headers resp)))
(current-upstream-hub (link-header-ref parsed-link-headers 'hub #f))
(current-upstream-topic
(link-header-ref parsed-link-headers 'self (current-upstream-topic)))
(define new-content-hash (sha1 (open-input-bytes response-body)))
(define new-content-type (dict-ref (web-response-header-headers resp) 'content-type #f))
(when (not (and (equal? (current-content-hash) new-content-hash)
(equal? (current-content-type) new-content-type)))
(current-content-hash new-content-hash)
(current-content-type new-content-type)
(send! (notification requested-topic
(current-upstream-hub)
(current-upstream-topic)
response-body
new-content-type)))]
[other
(log-warning "Upstream ~s (original: ~s) yielded ~a code ~a during poll"
poll-upstream-topic
requested-topic
other
(web-response-header-code resp))])
(poll-busy? #f)))
(on-start (poll-upstream!)) ;; always check at least once, for discovery
(on #:when (poll-interval-seconds) (asserted (later-than (+ (last-upstream-check)
(* 1000
(or (poll-interval-seconds) 0)))))
(poll-upstream!))
(assert #:when (current-upstream-hub)
(upstream-link requested-topic (current-upstream-hub) (current-upstream-topic)))
(on (message (notification requested-topic
$new-hub
$new-topic
$new-body
$new-type))
(current-upstream-hub new-hub)
(current-upstream-topic new-topic)
(current-content-hash (sha1 (open-input-bytes new-body)))
(current-content-type new-type)))
(define (upstream-link-main requested-topic upstream-hub upstream-topic)
(define sub-id (random-hex-string 16))
(on-start
(log-info "Starting remote sub endpoint ~a for requested topic ~s at hub ~s, topic ~s"
sub-id requested-topic upstream-hub upstream-topic))
(on-stop
(log-info "Stopping remote sub endpoint ~a for requested topic ~s at hub ~s, topic ~s"
sub-id requested-topic upstream-hub upstream-topic))
(field [next-subscription-refresh #f])
(begin/dataflow
(when (next-subscription-refresh)
(log-info "Next subscription refresh for topic ~s will occur in ~a seconds."
requested-topic
(/ (- (next-subscription-refresh) (current-inexact-milliseconds)) 1000.0))))
(define/query-config subscription-retry-delay 600)
Reinterpret canonical-local-host -> canonical-baseurl. In order to support HTTPS nginx proxying, we need to maintain a "fictional" baseurl which is not directly connected to this server's listen ports. This has caused a change to configuration: now, a baseurl is specified along with one or more listen specifications.
2016-11-21 22:35:39 +00:00
(during (canonical-baseurl $baseurl)
(define callback (canonical-url baseurl `("sub" (,sub-id ()))))
Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
(define (refresh-subscription!)
;; TODO: shared secret
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(log-info "Subscribing endpoint ~a to hub ~s for topic ~s (originally requested as ~s)"
sub-id
upstream-hub
upstream-topic
requested-topic)
(analyze-response
(lambda () (web-post/form-parameters! upstream-hub `((hub.callback . ,callback)
(hub.mode . "subscribe")
(hub.topic . ,upstream-topic))))
void
(lambda (resp response-body)
(log-warning
"Upstream subscription to hub ~s for topic ~s failed:\n - headers: ~v\n - body: ~s"
upstream-hub
upstream-topic
resp
response-body)
(next-subscription-refresh (+ (current-inexact-milliseconds)
(* 1000.0 (subscription-retry-delay)))))))
Split server.rkt into hub.rkt and hub/*.rkt
2016-11-20 21:13:40 +00:00
(define (unsubscribe!)
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(log-info "Unsubscribing endpoint ~a from hub ~s for topic ~s (originally requested as ~s)"
sub-id
upstream-hub
upstream-topic
requested-topic)
(analyze-response
(lambda () (web-post/form-parameters! upstream-hub `((hub.callback . ,callback)
(hub.mode . "unsubscribe")
(hub.topic . ,upstream-topic))))
void
(lambda (resp response-body)
(log-warning
"Upstream unsubscription from hub ~s for topic ~s failed:\n - headers: ~v\n - body: ~s"
upstream-hub
upstream-topic
resp
response-body))))
(on-start (refresh-subscription!))
Fixes and improvements toward proper upstream subscriptions
2016-11-21 21:04:07 +00:00
(on-stop (unsubscribe!))
(on #:when (next-subscription-refresh) (asserted (later-than (next-subscription-refresh)))
(next-subscription-refresh #f)
(refresh-subscription!))
Reinterpret canonical-local-host -> canonical-baseurl. In order to support HTTPS nginx proxying, we need to maintain a "fictional" baseurl which is not directly connected to this server's listen ports. This has caused a change to configuration: now, a baseurl is specified along with one or more listen specifications.
2016-11-21 22:35:39 +00:00
(during (http-listener $host-name $port)
(on (web-request-get (id req) (vh host-name port) ("sub" (,sub-id ())))
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(log-info "~a received verification-of-intent: ~v" sub-id (web-request-header-query req))
Reinterpret canonical-local-host -> canonical-baseurl. In order to support HTTPS nginx proxying, we need to maintain a "fictional" baseurl which is not directly connected to this server's listen ports. This has caused a change to configuration: now, a baseurl is specified along with one or more listen specifications.
2016-11-21 22:35:39 +00:00
(define challenge (dict-ref (web-request-header-query req) 'hub.challenge ""))
Convert the issued lease-seconds from string
2016-11-21 22:51:10 +00:00
(define lease-seconds-str (dict-ref (web-request-header-query req) 'hub.lease_seconds #f))
(if lease-seconds-str
Reinterpret canonical-local-host -> canonical-baseurl. In order to support HTTPS nginx proxying, we need to maintain a "fictional" baseurl which is not directly connected to this server's listen ports. This has caused a change to configuration: now, a baseurl is specified along with one or more listen specifications.
2016-11-21 22:35:39 +00:00
(next-subscription-refresh (+ (current-inexact-milliseconds)
Convert the issued lease-seconds from string
2016-11-21 22:51:10 +00:00
(* 1000.0
(shrink-lease (string->number lease-seconds-str)))))
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(log-warning "Hub ~s, topic ~s did not supply hub.lease_seconds"
upstream-hub
upstream-topic))
Reinterpret canonical-local-host -> canonical-baseurl. In order to support HTTPS nginx proxying, we need to maintain a "fictional" baseurl which is not directly connected to this server's listen ports. This has caused a change to configuration: now, a baseurl is specified along with one or more listen specifications.
2016-11-21 22:35:39 +00:00
(web-respond/bytes! id (string->bytes/utf-8 challenge)))
(on (web-request-incoming (id req) (vh host-name port) 'post ("sub" (,sub-id ())) $body)
;; TODO: verify the use of the shared secret
Remove layer of actor*, to avoid out-of-context field access; switch from 201 to 202 for posts
2016-11-22 00:32:32 +00:00
(define parsed-link-headers (parse-link-headers (web-request-header-headers req)))
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(define new-hub (link-header-ref parsed-link-headers 'hub #f))
(define new-topic (link-header-ref parsed-link-headers 'self #f))
Remove layer of actor*, to avoid out-of-context field access; switch from 201 to 202 for posts
2016-11-22 00:32:32 +00:00
(define content-type (web-request-header-content-type req))
(web-respond/status! id 202 #"Accepted")
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
(log-info "~a topic ~s got ~v message ~v; upstream hub ~s, topic ~s"
sub-id
requested-topic
Remove layer of actor*, to avoid out-of-context field access; switch from 201 to 202 for posts
2016-11-22 00:32:32 +00:00
content-type
body
Split out relay subscriptions from remote topics. The previous code was trying to do too much with local dataflow. This refactoring splits (stateful) relay upstream subscriptions out into actors of their own. Previously, dataflow variables were used to hold a single upstream hub and a single upstream canonical topic URL. This causes problems when things change rapidly: an unsubscription or subscription might be in flight, and there was no way to tell what the resulting state of the remote topic as a whole should be if the fields had changed between the start and finish of an in flight action. The new design follows from the observation that an individual subscription to an upstream hub is a stateful entity: it deserves promotion to a full actor. There is now an actor for the remote topic, and one for each established upstream subscription. Upstream subscriptions are created in response to demand expressed by the remote topic actor; to switch, all it needs to do is update its demand, and new subscriptions will spring into being while surplus subscriptions gracefully terminate. Also, this commit fixes a bug where upstream subscriptions were being made to the requested-topic rather than to the upstream-topic. Consequently, self-subscriptions (with multiple `http-listener` aliases) work now without infinite mail loops. There is still an issue that blocking actions in on-stop clauses don't run to completion; that is being dealt with separately, as part of a reexamination of facet-terminating events more generally.
2016-11-22 02:16:35 +00:00
new-hub
new-topic)
(send! (notification requested-topic
new-hub
new-topic
Remove layer of actor*, to avoid out-of-context field access; switch from 201 to 202 for posts
2016-11-22 00:32:32 +00:00
body
content-type))))))