Background and overview.
This commit is contained in:
parent
14ff172621
commit
fce69a7d57
|
@ -0,0 +1,102 @@
|
||||||
|
#lang scribble/manual
|
||||||
|
@require[racket/include]
|
||||||
|
@include{prelude.inc}
|
||||||
|
|
||||||
|
@title{Background}
|
||||||
|
|
||||||
|
@section{Network programming in the small}
|
||||||
|
|
||||||
|
Even the simplest of programs can be seen as a network. This program
|
||||||
|
sends a message containing @racket[3] and @racket[4] to @racket[+],
|
||||||
|
and awaits a reply message, which contains @racket[7]:
|
||||||
|
|
||||||
|
@interaction[(+ 3 4)]
|
||||||
|
|
||||||
|
Subroutines are network services. Here, when the message @racket[3] is
|
||||||
|
sent to @racket[add-four-to], it delegates to @racket[+]:
|
||||||
|
|
||||||
|
@interaction[(define (add-four-to n) (+ n 4))
|
||||||
|
(add-four-to 3)]
|
||||||
|
|
||||||
|
@note{In π-calculus encodings of programs, replication and state are
|
||||||
|
made explicit.} Immutable programs (of which stateless programs are a
|
||||||
|
special case) are easily able to be @emph{replicated}; programs using
|
||||||
|
mutable state, however, must be fixed in place. In fact, strictly
|
||||||
|
speaking, it is the @emph{state} that must be fixed in place, not the
|
||||||
|
program @emph{manipulating} the state. In the following program, the
|
||||||
|
mutable @racket[total] variable must not be replicated:
|
||||||
|
|
||||||
|
@interaction[(define accumulate
|
||||||
|
(let ((total 0))
|
||||||
|
(lambda (n)
|
||||||
|
(set! total (+ total n))
|
||||||
|
total)))
|
||||||
|
(accumulate 3)
|
||||||
|
(accumulate 4)
|
||||||
|
(accumulate 0)]
|
||||||
|
|
||||||
|
It would be a mistake to simply replace each occurrence of
|
||||||
|
@racket[accumulate] with its definition, since three separate
|
||||||
|
@racket[total] locations would be created. However, there is no
|
||||||
|
problem replicating the @racket[lambda] term, so long as
|
||||||
|
@racket[total] always refers to the same location:
|
||||||
|
|
||||||
|
@interaction[(define total 0)
|
||||||
|
((lambda (n) (set! total (+ total n)) total) 3)
|
||||||
|
((lambda (n) (set! total (+ total n)) total) 4)
|
||||||
|
((lambda (n) (set! total (+ total n)) total) 0)]
|
||||||
|
|
||||||
|
Programs raise exceptions to signal partial failure:
|
||||||
|
|
||||||
|
@interaction[(define accumulate
|
||||||
|
(let ((total 0))
|
||||||
|
(lambda (n)
|
||||||
|
(when (negative? n)
|
||||||
|
(error 'accumulate "n must be non-negative!"))
|
||||||
|
(set! total (+ total n))
|
||||||
|
total)))
|
||||||
|
(accumulate 3)
|
||||||
|
(accumulate -2)]
|
||||||
|
|
||||||
|
Programs can handle exceptions to @emph{contain} partial failure:
|
||||||
|
|
||||||
|
@interaction[(with-handlers ([exn:fail? (lambda (e) 'ok)])
|
||||||
|
(error 'oh-dear))]
|
||||||
|
|
||||||
|
Partial failures interact with shared, mutable state in unpredictable
|
||||||
|
ways, especially in larger programs that combine stateful subprograms.
|
||||||
|
|
||||||
|
Programmers often ignore issues of security and trust between
|
||||||
|
subprograms within a larger program. Most programming languages
|
||||||
|
entirely lack any means of securely isolating subprograms from each
|
||||||
|
other, leading to predictable failures. Even memory-safe languages
|
||||||
|
such as Racket, Java and .NET only offer weak techniques for securely
|
||||||
|
composing mutually suspicious subprograms. Techniques such as avoiding
|
||||||
|
@emph{ambient authority} and using @emph{object capabilities} to
|
||||||
|
compose subprograms
|
||||||
|
|
||||||
|
TODO ^
|
||||||
|
|
||||||
|
@section{Network programming in the large}
|
||||||
|
|
||||||
|
Programs which engage in I/O are very obviously part of a network:
|
||||||
|
|
||||||
|
@racketblock[(define username (read-line))
|
||||||
|
(printf "Hello, ~a!\n" username)]
|
||||||
|
|
||||||
|
But look closely! There is a difference here between the kind of communication
|
||||||
|
|
||||||
|
This program is more obviously composed of a number of moving pieces:
|
||||||
|
|
||||||
|
@interaction[(let ((b (box 3)))
|
||||||
|
(set-box! b (+ (unbox b) 4))
|
||||||
|
(unbox b))]
|
||||||
|
|
||||||
|
@interaction[(define (add-three-to b)
|
||||||
|
(set-box! b (+ (unbox b) 3)))
|
||||||
|
(define (add-four-to b)
|
||||||
|
(set-box! b (+ (unbox b) 4)))
|
||||||
|
(define my-box (box 0))
|
||||||
|
(add-three-to my-box)
|
||||||
|
(add-four-to my-box)
|
||||||
|
(unbox my-box)]
|
|
@ -4,25 +4,13 @@
|
||||||
|
|
||||||
@title{Concepts}
|
@title{Concepts}
|
||||||
|
|
||||||
In this paper, we present a novel "marketplace" approach to
|
We integrate ideas from both distributed systems
|
||||||
functional systems programming, generalizing
|
|
||||||
Felleisen et al.'s "Worlds and Universes" approach to
|
|
||||||
functional I/O
|
|
||||||
[ICFP 2009]. We integrate ideas from both distributed systems
|
|
||||||
and virtualized operating system designs to obtain a novel
|
and virtualized operating system designs to obtain a novel
|
||||||
architecture of nested virtual machines. Each nested layer is equipped with
|
architecture of nested virtual machines. Each nested layer is equipped with
|
||||||
its own publish/subscribe network that also propagates information
|
its own publish/subscribe network that also propagates information
|
||||||
about the (dis)appearance of services. Our paper presents
|
about the (dis)appearance of services.
|
||||||
several case studies, including a recursive DNS resolver that
|
|
||||||
has served our lab's DNS traffic for the past nine months.
|
|
||||||
|
|
||||||
Our goal is to reconcile this tension, starting from the "Worlds
|
The result suggests a @emph{
|
||||||
and Universes" approach to functional I/O and
|
|
||||||
generalizing to
|
|
||||||
functional systems programming. We augment
|
|
||||||
its inherent support for concurrency with publish/subscribe (pub/sub)
|
|
||||||
messaging, a notion of presence, and
|
|
||||||
nestable virtual machines (VMs). The result suggests a @emph{
|
|
||||||
marketplace} metaphor, where communicating programs exist in a
|
marketplace} metaphor, where communicating programs exist in a
|
||||||
noisy, crowded, even chaotic context, rather than in a quiet place
|
noisy, crowded, even chaotic context, rather than in a quiet place
|
||||||
systematically going through their inboxes.
|
systematically going through their inboxes.
|
||||||
|
|
|
@ -2,16 +2,71 @@
|
||||||
@require[racket/include]
|
@require[racket/include]
|
||||||
@include{prelude.inc}
|
@include{prelude.inc}
|
||||||
|
|
||||||
@title[#:tag "marketplace"]{Marketplace: A Functional Operating System}
|
@title[#:tag "marketplace"]{Marketplace: Network-Aware Programming}
|
||||||
|
@;{Marketplace: A Functional Operating System}
|
||||||
|
@;{Marketplace: A Functional Network Operating System}
|
||||||
|
@;{Marketplace: A Functional Distributed Operating System}
|
||||||
|
|
||||||
@author[(author+email "Tony Garnock-Jones" "tonyg@ccs.neu.edu")]
|
@author[(author+email "Tony Garnock-Jones" "tonyg@ccs.neu.edu")]
|
||||||
|
|
||||||
@defmodulelang[marketplace]
|
@bold{Every program is a network.}
|
||||||
|
This is the insight behind the π-calculus. Encoding a program as a
|
||||||
|
π-calculus term shows it as a network of communicating processes. It
|
||||||
|
is also one of the original inspirations for Smalltalk, where every
|
||||||
|
object, every value, was imagined to be a separate computer in a vast
|
||||||
|
network, and where objects communicated by message-passing.
|
||||||
|
|
||||||
This manual TODO (overview & motivation)
|
@bold{Every program is part of a network.}
|
||||||
|
A program that computes a result but cannot communicate it is useless
|
||||||
|
indeed. Every complete program both @emph{computes} and
|
||||||
|
@emph{communicates}. Furthermore, it does so with some finite set of
|
||||||
|
@emph{resources}, which it must carefully manage.
|
||||||
|
|
||||||
|
@bold{Our programming languages do not recognise that every program is
|
||||||
|
a network.} They blur the distinction between stateful and stateless
|
||||||
|
portions of a program, making it difficult for programmers to reason
|
||||||
|
about concurrency, contention, and distribution. They often treat
|
||||||
|
partial failure as an afterthought, despite its importance in
|
||||||
|
reasoning about program behaviour, particularly in connection with the
|
||||||
|
effect of exceptions on stateful programs. They seldom consider issues
|
||||||
|
of trust and security.
|
||||||
|
|
||||||
|
@bold{Our programming languages do not recognise that every program is part of a network.}
|
||||||
|
They treat communication with the outside world in an ad-hoc manner.
|
||||||
|
They frequently treat network communication separately from
|
||||||
|
@tt{1950s-style terminal input and output}. They force the programmer
|
||||||
|
to divine failures in other parts of the network by arcane means such
|
||||||
|
as timeouts and examining the entrails of dead communication channels.
|
||||||
|
They offer no support for allocating or releasing local resources in
|
||||||
|
response to changes in other parts of the network. They seldom
|
||||||
|
consider issues of trust and security.
|
||||||
|
|
||||||
|
@bold{Marketplace is a network-aware programming language.} As a
|
||||||
|
corollary, because every program not only computes but also
|
||||||
|
communicates and manages its resources, Marketplace is also a
|
||||||
|
distributed operating system.
|
||||||
|
|
||||||
|
By recognising that programs communicate both
|
||||||
|
@emph{internally} (between subprograms) and @emph{externally} (between
|
||||||
|
peers), we recognise an inherently recursive layered architecture. We
|
||||||
|
see at every level the @emph{same} concerns of resource management,
|
||||||
|
location of mutable state, failure detection and recovery, access
|
||||||
|
control, I/O and user interface, debugging and profiling.
|
||||||
|
|
||||||
|
Marketplace addresses these concerns with a small set of primitives
|
||||||
|
chosen to make network programming in-the-small as flexible, scalable,
|
||||||
|
manageable and securable as network programming in-the-large---and
|
||||||
|
vice versa.
|
||||||
|
|
||||||
|
@;{
|
||||||
|
Networks must be manageable. Networks must be monitorable. Networks
|
||||||
|
must tolerate partial failure. Networks must scale. Networks must
|
||||||
|
communicate with other networks, via yet other networks.
|
||||||
|
}
|
||||||
|
|
||||||
@local-table-of-contents[]
|
@local-table-of-contents[]
|
||||||
|
|
||||||
|
@include-section["background.scrbl"]
|
||||||
@include-section["concepts.scrbl"]
|
@include-section["concepts.scrbl"]
|
||||||
@include-section["highlevel.scrbl"]
|
@include-section["highlevel.scrbl"]
|
||||||
@include-section["lowlevel.scrbl"]
|
@include-section["lowlevel.scrbl"]
|
||||||
|
|
|
@ -1,5 +1,6 @@
|
||||||
;; -*- scheme -*-
|
;; -*- scheme -*-
|
||||||
(require scribble/racket
|
(require scribble/racket
|
||||||
|
scribble/eval
|
||||||
scriblib/footnote
|
scriblib/footnote
|
||||||
(for-syntax racket)
|
(for-syntax racket)
|
||||||
(for-label typed/racket/base))
|
(for-label typed/racket/base))
|
||||||
|
|
Loading…
Reference in New Issue