1528 lines
52 KiB
Markdown
1528 lines
52 KiB
Markdown
Nix-based process management framework
|
|
======================================
|
|
This repository contains a very experimental prototype implementation of an
|
|
operating system and process manager agnostic Nix-based process managed
|
|
framework that can be used to run multiple instances of services on a single
|
|
machine, using Nix to deliver and isolate all required package dependencies and
|
|
configuration files.
|
|
|
|
Features:
|
|
* It uses *simple conventions* to specify system configurations: function
|
|
definitions (corresponding to constructors), function invocations (that
|
|
compose running process instances from constructors) and Nix profiles (that
|
|
assembles multiple process configurations into one package).
|
|
* It identifies process dependencies, so that a process manager can ensure that
|
|
processes are activated and deactivated in the right order.
|
|
* The ability to deploy *multiple instances* of the same process, by making
|
|
conflicting resources configurable.
|
|
* Deploying processes/services as an *unprivileged user*.
|
|
* Operating system and process manager *agnostic* -- it can be used on any
|
|
operating system that supports the Nix package manager and works with a
|
|
variety of process managers.
|
|
* Advanced concepts and features, such as namespaces and cgroups, are not
|
|
required.
|
|
|
|
Supported process managers
|
|
==========================
|
|
Currently, the following process managers are supported:
|
|
|
|
* `sysvinit`: sysvinit scripts, also known as [LSB Init scripts](https://wiki.debian.org/LSBInitScripts)
|
|
* `bsdrc`: BSD [rc scripts](https://www.freebsd.org/doc/en_US.ISO8859-1/articles/rc-scripting/index.html)
|
|
* `supervisord`: [supervisord](http://supervisord.org)
|
|
* `systemd`: [systemd](https://www.freedesktop.org/wiki/Software/systemd)
|
|
* `launchd`: [launchd](https://www.launchd.info)
|
|
* `cygrunsrv`: Cygwin's [cygrunsrv](http://web.mit.edu/cygwin/cygwin_v1.3.2/usr/doc/Cygwin/cygrunsrv.README)
|
|
* `s6-rc`: [s6-rc](https://skarnet.org/software/s6-rc) for managing services
|
|
supervised by [s6](https://skarnet.org/software/s6)
|
|
* `synit`: [Synit](https://synit.org/)
|
|
|
|
It can also work with the following solutions that are technically not
|
|
categorized as process managers (but can still be used as such):
|
|
|
|
* `docker`: [Docker](https://docker.com) is technically more than just a process
|
|
manager, but by sharing the host's network, Nix store, and bind mounting
|
|
relevant state directories, it can also serve as a process manager with
|
|
similar functionality as the others described above.
|
|
* `disnix`: Technically [Disnix](https://github.com/svanderburg/disnix) is not
|
|
a process manager but it is flexible enough to start daemons and arrange
|
|
activation ordering. This target backend is not designed for managing
|
|
production systems, but it is quite convenient as a simple solution
|
|
for experimentation that is supported on most UNIX-like systems.
|
|
|
|
Prerequisites
|
|
=============
|
|
To use this framework, you first need to install:
|
|
|
|
* [The Nix package manager](http://nixos.org/nix)
|
|
* [The Nixpkgs collection](http://nixos.org/nixpkgs)
|
|
* [Dysnomia](http://github.com/svanderburg/dysnomia), if you want to manage users and groups
|
|
* To use the ID assigner tool: `nixproc-id-assign` (for port numbers, UIDs and
|
|
GIDs), you need a recent development version of
|
|
[Dynamic Disnix](https://github.com/svanderburg/dydisnix)
|
|
|
|
Installation
|
|
============
|
|
First, make a Git clone of this repository.
|
|
|
|
The next step is installing the common tools:
|
|
|
|
```bash
|
|
$ cd tools
|
|
$ nix-env -f default.nix -iA common
|
|
```
|
|
|
|
Then, at least one tool that deploys a configuration for a supported process
|
|
manager must be installed.
|
|
|
|
For example, to work with sysvinit scripts, you must install:
|
|
|
|
```bash
|
|
$ nix-env -f default.nix -iA sysvinit
|
|
```
|
|
|
|
To work with a different process manager, you should replace `sysvinit` with
|
|
any of the supported process managers listed in the previous section. For
|
|
example, to use utilities to generate and deploy `systemd` configurations, you
|
|
should run:
|
|
|
|
```bash
|
|
$ nix-env -f default.nix -iA systemd
|
|
```
|
|
|
|
Usage
|
|
=====
|
|
For each kind of process you need to write a Nix expression that constructs
|
|
its configuration from sources and its dependencies. This can be done in a
|
|
process manager-specific and a process manager-agnostic way.
|
|
|
|
Then you need to create a constructors and processes expression.
|
|
|
|
The processes expression can be used by a deploy tool that works with a
|
|
specific process manager.
|
|
|
|
Writing a process manager-specific process management configuration
|
|
-------------------------------------------------------------------
|
|
The following expression is an example of a configuration that deploys
|
|
a sysvinit script that can be used to control a simple web application
|
|
process (with an embedded HTTP server) that just returns a static HTML page:
|
|
|
|
```nix
|
|
{createSystemVInitScript, tmpDir}:
|
|
|
|
let
|
|
webapp = import ../../webapp;
|
|
user = "webapp";
|
|
group = "webapp";
|
|
in
|
|
createSystemVInitScript {
|
|
name = "webapp";
|
|
process = "${webapp}/bin/webapp";
|
|
args = [ "-D" ];
|
|
environment = {
|
|
PORT = 5000;
|
|
PID_FILE = "${tmpDir}/webapp.pid";
|
|
};
|
|
|
|
runlevels = [ 3 4 5 ];
|
|
|
|
inherit user;
|
|
|
|
credentials = {
|
|
groups = {
|
|
"${group}" = {};
|
|
};
|
|
users = {
|
|
"${user}" = {
|
|
inherit group;
|
|
description = "Webapp";
|
|
};
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
A process expression defines a function:
|
|
|
|
* The function header (first line) allows build-time dependencies and common
|
|
configuration properties to be configured, such as the the function that
|
|
constructs sysvinit scripts (`createSystemVInitScript`) and the `runtimeDir`
|
|
in which PID files are stored (on most systems this defaults to: `/var/run`).
|
|
|
|
In the body, we invoke the `createSystemVInitScript` function to declaratively
|
|
construct a sysvinit script:
|
|
|
|
* The `process` parameter specifies which process should be managed. From this
|
|
parameter, the generator will automatically derive `start`, `stop`, `restart`
|
|
and `reload` activities.
|
|
* The `args` parameter specifies the command line parameters propagated to the
|
|
process. The `-D` parameter specifies that the webapp process should run in
|
|
daemon mode (i.e. the process spawns another process that keeps running in
|
|
the background and then terminates immediately).
|
|
* The `environment` attribute set defines all environment variables that the
|
|
webapp process needs -- `PORT` is used to specify the TCP port it should
|
|
listen to and `PID_FILE` specifies the path to the PID file that stores
|
|
the process ID (PID) of the daemon process
|
|
* The `runlevels` parameter specifies in which run levels the process should
|
|
be started (in the above example: 3, 4, and 5. It will automatically
|
|
configure the init system to stop the process in the other runlevels:
|
|
0, 1, 2, and 6.
|
|
* It is also typically not recommended to run a service as root user. The
|
|
`credentials` attribute specifies which group and user account need to be
|
|
created. The `user` parameter specifies the user the process needs to
|
|
run as.
|
|
|
|
The `createSystemVInitScript` function supports many more parameters than
|
|
described in the example above. For example, it is also possible to directly
|
|
specify how activities should be implemented.
|
|
|
|
It can also be used to specify `dependencies` on other sysvinit scripts -- the
|
|
system will automatically derive the sequence numbers so that they are activated
|
|
and deactivated in the right order.
|
|
|
|
In addition to sysvinit, there are also functions that can be used to create
|
|
configurations for the other supported process managers, e.g.
|
|
`createSystemdUnit`, `createSupervisordProgram`, `createBSDRCScript`. Check
|
|
the implementations in `nixproc/backends` for more information.
|
|
|
|
Writing an instantiatable process configuration
|
|
-----------------------------------------------
|
|
The example shown earlier only allows you to deploy a single instance of the
|
|
`webapp` process. A second instance cannot co-exist because it will allocate
|
|
conflicting resources, such as the TCP port it binds to and the PID file.
|
|
These resources can only be assigned once.
|
|
|
|
It is also possible to revise the example to allow multiple process instances
|
|
to co-exist, by making conflicting resources configurable.
|
|
|
|
An instantiatable process expression defines a nested function:
|
|
|
|
```nix
|
|
{createSystemVInitScript, tmpDir}:
|
|
{port, instanceSuffix ? "", instanceName ? "webapp${instanceSuffix}"}:
|
|
|
|
let
|
|
webapp = import ../../webapp;
|
|
in
|
|
createSystemVInitScript {
|
|
inherit instanceName;
|
|
process = "${webapp}/bin/webapp";
|
|
args = [ "-D" ];
|
|
environment = {
|
|
PORT = port;
|
|
PID_FILE = "${tmpDir}/${instanceName}.pid";
|
|
};
|
|
|
|
runlevels = [ 3 4 5 ];
|
|
|
|
user = instanceName;
|
|
|
|
credentials = {
|
|
groups = {
|
|
"${instanceName}" = {};
|
|
};
|
|
users = {
|
|
"${instanceName}" = {
|
|
group = instanceName;
|
|
description = "Webapp";
|
|
};
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
* The outer function (first line) allows properties to be configured that apply
|
|
to all process instances, such as the the function that constructs sysvinit
|
|
scripts (`createSystemVInitScript`) and the `runtimeDir` in which PID files
|
|
are stored (on most systems this defaults to: `/var/run`).
|
|
* The inner function (second line) refers to all instance specific parameters.
|
|
To allow multiple instances to co-exist, instance parameters must be
|
|
configured in such a way that they no longer conflict. For example, if we
|
|
assign two unique TCP `port` numbers and we append the process name with a
|
|
unique suffix, we can run two instances of the web application at the same
|
|
time.
|
|
* The process should have a unique name to identify it with. If no `name`
|
|
parameter was specified, then the `name` will automatically correspond to
|
|
`instanceName`.
|
|
* We must make sure that each has a unique PID file name. We can use the
|
|
`instanceName` parameter to specify what name this PID file should have.
|
|
By default, the PID file gets the same name as the process instance name.
|
|
* To also allocate a unique user and group for the process. We are using the
|
|
`instanceName` parameter as a unique user and group name.
|
|
|
|
Writing a process manager-agnostic process management configuration
|
|
-------------------------------------------------------------------
|
|
This repository contains generator functions for a variety of process managers.
|
|
What you will notice is that they accept parameters that look quite similar.
|
|
|
|
When it is desired to target multiple process managers, it is also possible to
|
|
write a process manager-agnostic configuration from which configuration files
|
|
can be generated for all supported process management backends.
|
|
|
|
This is a process manager-agnostic version of the previous example:
|
|
|
|
```nix
|
|
{createManagedProcess, tmpDir}:
|
|
{port, instanceSuffix ? "", instanceName ? "webapp${instanceSuffix}"}:
|
|
|
|
let
|
|
webapp = import ../../webapp;
|
|
in
|
|
createManagedProcess {
|
|
inherit instanceName;
|
|
description = "Simple web application";
|
|
|
|
# This expression can both run in foreground or daemon mode.
|
|
# The process manager can pick which mode it prefers.
|
|
process = "${webapp}/bin/webapp";
|
|
daemonArgs = [ "-D" ];
|
|
|
|
environment = {
|
|
PORT = port;
|
|
PID_FILE = "${tmpDir}/${instanceName}.pid";
|
|
};
|
|
user = instanceName;
|
|
credentials = {
|
|
groups = {
|
|
"${instanceName}" = {};
|
|
};
|
|
users = {
|
|
"${instanceName}" = {
|
|
group = instanceName;
|
|
description = "Webapp";
|
|
};
|
|
};
|
|
};
|
|
|
|
overrides = {
|
|
sysvinit = {
|
|
runlevels = [ 3 4 5 ];
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
In the above example, we invoke `createManagedProcess` to construct a
|
|
configuration for any process manager supported by this framework. It captures
|
|
similar properties that are described in the sysvinit-specific configuration,
|
|
as shown in the previous example.
|
|
|
|
To allow the specification to target a variety of process managers, we must
|
|
specify:
|
|
|
|
* How the process can be started in foreground and daemon mode. The `process`
|
|
parameter gets translated to `foregroundProcess` and `daemon`. The former
|
|
specifies how the service should be started as a foreground process and the
|
|
latter how it should start as a daemon.
|
|
* The `daemonArgs` parameter specifies which command-line parameters the process
|
|
should take when it is supposed to run as a daemon.
|
|
|
|
Under the hood, the `createManagedProcess` function invokes a generator function
|
|
that calls the corresponding process manager-specific create function.
|
|
|
|
The `createManagedProcess` abstraction function does not support all
|
|
functionality that the process manager-specific abstraction functions provide --
|
|
it only supports a common subset. To get non-standardized functionality working,
|
|
you can also define `overrides`, that augment the generated function parameters
|
|
with process manager-specific parameters.
|
|
|
|
In the above example, we define an override to specify the `runlevels`. Runlevels
|
|
is a concept only supported by sysvinit scripts.
|
|
|
|
Defining process manager-specific overrides
|
|
-------------------------------------------
|
|
As described in the previous section, the `createManagedProcess` abstraction only
|
|
works with high-level concepts that are easily generalizable to all kinds of
|
|
process managers.
|
|
|
|
The attribute set of parameters passed to the `createManagedProcess` function
|
|
gets translated to an attribute set of parameters for the corresponding
|
|
process manager-specific abstraction functions, e.g. `createSystemVInitScript`,
|
|
`createSupervisordProgram`, `createSystemdService` etc.
|
|
|
|
We can change the content of the generated attribute set, allowing you to get
|
|
access to any property of a process manager backend including properties for
|
|
which the `createManagedProcess` function does provide any high-level concepts.
|
|
|
|
An override can be be an attribute set that simply overrides or augments the
|
|
process manager-specific parameter attribute set:
|
|
|
|
```nix
|
|
{createManagedProcess, tmpDir}:
|
|
{port, instanceSuffix ? "", instanceName ? "webapp${instanceSuffix}"}:
|
|
|
|
let
|
|
webapp = import ../../webapp;
|
|
in
|
|
createManagedProcess {
|
|
inherit instanceName;
|
|
description = "Simple web application";
|
|
|
|
# This expression can both run in foreground or daemon mode.
|
|
# The process manager can pick which mode it prefers.
|
|
process = "${webapp}/bin/webapp";
|
|
daemonArgs = [ "-D" ];
|
|
|
|
environment = {
|
|
PORT = port;
|
|
PID_FILE = "${tmpDir}/${instanceName}.pid";
|
|
};
|
|
|
|
overrides = {
|
|
sysvinit.runlevels = [ 3 4 5 ];
|
|
systemd = {
|
|
Service.Restart = "always";
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
In the above example case, we use an override to define in which runlevels the
|
|
service should start (a sysvinit specific concept), and when systemd is used,
|
|
the service gets restarted automatically when it stops (which is not a universal
|
|
property all process managers support, but systemd does).
|
|
|
|
It is also possible to write an override as a function which is more powerful --
|
|
you can also delete and augment existing parameters with additional information,
|
|
if desired:
|
|
|
|
```nix
|
|
{createManagedProcess, tmpDir}:
|
|
{port, instanceSuffix ? "", instanceName ? "webapp${instanceSuffix}"}:
|
|
|
|
let
|
|
webapp = import ../../webapp;
|
|
in
|
|
createManagedProcess {
|
|
inherit instanceName;
|
|
description = "Simple web application";
|
|
|
|
# This expression can both run in foreground or daemon mode.
|
|
# The process manager can pick which mode it prefers.
|
|
process = "${webapp}/bin/webapp";
|
|
daemonArgs = [ "-D" ];
|
|
|
|
environment = {
|
|
PORT = port;
|
|
PID_FILE = "${tmpDir}/${instanceName}.pid";
|
|
};
|
|
|
|
overrides = {
|
|
sysvinit.runlevels = [ 3 4 5 ];
|
|
systemd = systemdArgs: systemdArgs // {
|
|
Service = systemdArgs.Service // {
|
|
ExecStart = "${systemdArgs.Service.ExecStart} -D";
|
|
Type = "forking";
|
|
};
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
In the above example, I modify the generated systemd arguments in such a way
|
|
that the service runs in daemon mode and it is managed as a daemon (by default,
|
|
the systemd generator prefers to work with foreground processes).
|
|
|
|
Writing a constructors expression
|
|
---------------------------------
|
|
As shown in the previous sections, a process configuration is a nested function.
|
|
To be able to deploy a certain process configuration, it needs to be composed
|
|
twice.
|
|
|
|
The common parameters (the outer function) are composed in a so-called
|
|
constructors expression, that has the following structure:
|
|
|
|
```nix
|
|
{ pkgs
|
|
, stateDir
|
|
, logDir
|
|
, runtimeDir
|
|
, tmpDir
|
|
, forceDisableUserChange
|
|
, processManager
|
|
}:
|
|
|
|
let
|
|
createManagedProcess = import ../../nixproc/create-managed-process/universal/create-managed-process-universal.nix {
|
|
inherit pkgs runtimeDir tmpDir forceDisableUserChange processManager;
|
|
};
|
|
in
|
|
{
|
|
webapp = import ./webapp.nix {
|
|
inherit createManagedProcess tmpDir;
|
|
};
|
|
|
|
nginx = import ./nginx-reverse-proxy.nix {
|
|
inherit createManagedProcess stateDir logDir runtimeDir forceDisableUserChange;
|
|
inherit (pkgs) stdenv writeTextFile nginx;
|
|
};
|
|
}
|
|
```
|
|
|
|
The above Nix expression defines a function that takes common state
|
|
configuration parameters that applies to all services:
|
|
|
|
* `pkgs` refers to the Nixpkgs collection
|
|
* `stateDir` refers to the base directory where all variable data needs to be
|
|
stored. The default on most systems is `/var`.
|
|
* `runtimeDir` refers to the base directory where all PID files are stored
|
|
* `tmpDir` refers to the base directory where all temp files are stored
|
|
* `forceDisableUserChange` can be used to globally switch of the creation of
|
|
users and groups and changing users.
|
|
* `processManager` specifies which process manager we want to use (when it is
|
|
desired to do process manager agnostic deployments).
|
|
|
|
In the body, the function returns an attribute set in which every value refers
|
|
to a constructor function that can be used to construct process instances.
|
|
|
|
The `webapp` attribute refers to a constructor function that can be used to
|
|
construct one or more running `webapp` processes. It takes the common parameters
|
|
that it requires as function arguments.
|
|
|
|
The constructors attribute facilitates multiple constructor functions.
|
|
`nginxReverseProxy` refers to a process configuration that launches the
|
|
[Nginx](http://nginx.com) HTTP server and configures it to act as a reverse
|
|
proxy for an arbitrary number of web application processes.
|
|
|
|
Writing a processes expression
|
|
------------------------------
|
|
The processes Nix expression makes it possible to construct one or more
|
|
instances of processes:
|
|
|
|
```nix
|
|
{ pkgs ? import <nixpkgs> { inherit system; }
|
|
, system ? builtins.currentSystem
|
|
, stateDir ? "/var"
|
|
, runtimeDir ? "${stateDir}/run"
|
|
, logDir ? "${stateDir}/log"
|
|
, tmpDir ? (if stateDir == "/var" then "/tmp" else "${stateDir}/tmp")
|
|
, forceDisableUserChange ? false
|
|
, processManager
|
|
}:
|
|
|
|
let
|
|
constructors = import ./constructors.nix {
|
|
inherit pkgs stateDir runtimeDir logDir tmpDir forceDisableUserChange processManager;
|
|
};
|
|
in
|
|
rec {
|
|
webapp1 = rec {
|
|
port = 5000;
|
|
dnsName = "webapp1.local";
|
|
|
|
pkg = constructors.webapp {
|
|
inherit port;
|
|
};
|
|
};
|
|
|
|
webapp2 = rec {
|
|
port = 5001;
|
|
dnsName = "webapp2.local";
|
|
|
|
pkg = constructors.webapp {
|
|
inherit port;
|
|
};
|
|
};
|
|
|
|
nginx = rec {
|
|
port = 8080;
|
|
|
|
pkg = constructors.nginxReverseProxy {
|
|
webapps = [ webapp1 webapp2 ];
|
|
inherit port;
|
|
} {};
|
|
};
|
|
}
|
|
```
|
|
|
|
The above Nix expression is a function in which the parameters (again) specify
|
|
common state configuration parameters. It makes a reference the constructors
|
|
expression shown in the previous example.
|
|
|
|
The function body returns an attribute set that defines three process
|
|
instances:
|
|
|
|
* `webapp1` is a web application process that listens on TCP port 5000
|
|
* `webapp2` is a web application process that listens on TCP port 5001
|
|
* `nginxReverseProxy` is an Nginx server that forwards requests to the
|
|
web application processes. If the virtual host is `webapp1.local` then the
|
|
first `webapp1` process responds, if the virtual host is `webapp2.local` then
|
|
the second process (`webapp2`) responds. Nginx listens on TCP port 8080.
|
|
|
|
Building a process configurations profile
|
|
-----------------------------------------
|
|
We can build all required packages and generate all configuration artifacts for
|
|
a specific process manager by running the following command:
|
|
|
|
```bash
|
|
$ nixproc-build --process-manager sysvinit processes.nix
|
|
result/
|
|
```
|
|
|
|
The above command generates sysvinit scripts and start and stop symlinks to
|
|
ensure that the webapp processes are started before the Nginx reverse proxy.
|
|
|
|
The `--process-manager` parameter can be changed to generate configuration files
|
|
for different process managers. For example, if we would use
|
|
`--process-manager systemd` then the resulting Nix profile contains a collection
|
|
of systemd unit configuration files.
|
|
|
|
Deploying a process configurations profile
|
|
------------------------------------------
|
|
In addition to generating configuration files that can be consumed by a process
|
|
manager, we can also invoke the process manager to deploy all process defined in
|
|
our process Nix expression.
|
|
|
|
The following command automatically starts all sysvinit scripts (and stop all
|
|
obsolete sysvinit scripts, in case of an upgrade):
|
|
|
|
```bash
|
|
$ nixproc-sysvinit-switch processes.nix
|
|
```
|
|
|
|
Consult the help pages of the corresponding process manager specific tools to
|
|
get a better understanding on how they work.
|
|
|
|
Changing the state directories
|
|
------------------------------
|
|
By default, the all processes use the `/var` directory as a base directory to
|
|
store all state. This location can be adjusted by using the `--state-dir`
|
|
parameter.
|
|
|
|
The following command deploys all process instances and stores their state in
|
|
`/home/sander/var`:
|
|
|
|
```bash
|
|
$ nixproc-sysvinit-switch --state-dir /home/sander/var processes.nix
|
|
```
|
|
|
|
Similarly, it is also possible to adjust the base locations of the runtime
|
|
files, log files and temp files, if desired.
|
|
|
|
Deploying process instances as an unprivileged user
|
|
---------------------------------------------------
|
|
It is also possible to do unprivileged user deployments. Unfortunately,
|
|
unprivileged users cannot create new groups and/or users or change permissions
|
|
of running processes.
|
|
|
|
To still allow unprivileged user deployments, user configuration and switching
|
|
can be globally disabled with the `--force-disable-user-change` parameter.
|
|
Then the `credentials` and `user` switching parameters are ignored.
|
|
|
|
The following command makes it possible to deploy all processes as an
|
|
unprivileged user:
|
|
|
|
```bash
|
|
$ nixproc-sysvinit-switch --state-dir /home/sander/var --force-disable-user-change processes.nix
|
|
```
|
|
|
|
Undeploying the system
|
|
----------------------
|
|
It may also be desired to completely undeploy a system when it is no longer
|
|
needed. The following command completely undeploys all previously deployed
|
|
processes:
|
|
|
|
```bash
|
|
$ nixproc-sysvinit-switch --undeploy
|
|
```
|
|
|
|
Assigning unique IDs to services
|
|
--------------------------------
|
|
As explained earlier, to ensure that multiple process instances have no
|
|
conflicts, they require unique process instance parameters.
|
|
|
|
One catagory of process parameters are unique numeric IDs, such as port
|
|
numbers, UIDs and GIDs. It is possible to manually assign them, but this process
|
|
can also be automated.
|
|
|
|
The following configuration file is an ID resources configuration file
|
|
(`idresources.nix`) that defines pools of numeric ID resources:
|
|
|
|
```nix
|
|
rec {
|
|
webappPorts = {
|
|
min = 5000;
|
|
max = 6000;
|
|
};
|
|
|
|
nginxPorts = {
|
|
min = 8080;
|
|
max = 9000;
|
|
};
|
|
|
|
uids = {
|
|
min = 2000;
|
|
max = 3000;
|
|
};
|
|
|
|
gids = uids;
|
|
}
|
|
```
|
|
|
|
The above ID resources configuration defines the following resources:
|
|
|
|
* The `webappPorts` is a pool of unique TCP port number assigned to the `webapp`
|
|
processes shown in the previous examples. These are unique numbers between
|
|
5000 and 6000.
|
|
* The `nginxPorts` is a pool of unique TCP port numbers assigned to `nginx`
|
|
instances. These are unique numbers between 8080 and 9000.
|
|
* `uids` specifies the range of unique user IDs (UIDs) between 2000 and 3000.
|
|
* `gids` specifies the range of unique group IDs (GIDs). They are identical to
|
|
the `uids`.
|
|
|
|
To use automatic ID assignments, the processes model (`processes.nix`) can be
|
|
augmented as follows:
|
|
|
|
```nix
|
|
{ pkgs ? import <nixpkgs> { inherit system; }
|
|
, system ? builtins.currentSystem
|
|
, stateDir ? "/var"
|
|
, runtimeDir ? "${stateDir}/run"
|
|
, logDir ? "${stateDir}/log"
|
|
, tmpDir ? (if stateDir == "/var" then "/tmp" else "${stateDir}/tmp")
|
|
, forceDisableUserChange ? false
|
|
, processManager
|
|
}:
|
|
|
|
let
|
|
ids = if builtins.pathExists ./ids.nix then (import ./ids.nix).ids else {};
|
|
|
|
constructors = import ./constructors.nix {
|
|
inherit pkgs stateDir runtimeDir logDir tmpDir forceDisableUserChange processManager ids;
|
|
};
|
|
in
|
|
rec {
|
|
webapp1 = rec {
|
|
port = ids.webappPorts.webapp1 or 0;
|
|
dnsName = "webapp1.local";
|
|
|
|
pkg = constructors.webapp {
|
|
inherit port;
|
|
};
|
|
|
|
requiresUniqueIdsFor = [ "uids" "gids" "webappPorts" ];
|
|
};
|
|
}
|
|
```
|
|
|
|
The above process model has the following changes:
|
|
|
|
* Every process instance is annotated with a `requiresUniqueIdsFor` attribute
|
|
that specifies for which resources the process instances requires unique IDs
|
|
* In the beginning of the expression we have imported the generated `ids.nix`
|
|
expression that contains all generated unique ID assignments. If the file
|
|
does not exists, an empty attribute set is returned.
|
|
* Every process instance consumes the unique IDs from the `ids` attribute set,
|
|
as opposed to using hardcoded values. The `webapp1` service uses a auto
|
|
generated port assignment. If no such assignment exists, it defaults to 0
|
|
to allow the processes model evaluation not to fail, for the initial IDs
|
|
assignment.
|
|
|
|
To automatically generate the ID assignments from an ID resources configuration
|
|
and processes model, you can run:
|
|
|
|
```bash
|
|
$ nixproc-id-assign --id-resources idresources.nix --output-file ids.nix processes.nix
|
|
```
|
|
|
|
The above command automatically assigns IDs to all processes that require them and writes
|
|
the output result to the `ids.nix` file. This file may look as follows:
|
|
|
|
```nix
|
|
{
|
|
"ids" = {
|
|
"gids" = {
|
|
"webapp" = 2001;
|
|
};
|
|
"nginxPorts" = {
|
|
};
|
|
"uids" = {
|
|
"webapp" = 2001;
|
|
};
|
|
"webappPorts" = {
|
|
"webapp" = 5000;
|
|
};
|
|
};
|
|
"lastAssignments" = {
|
|
"gids" = 2001;
|
|
"uids" = 2001;
|
|
"webappPorts" = 5000;
|
|
};
|
|
}
|
|
```
|
|
|
|
The above expression defines two attributes:
|
|
* The `ids` attribute contains for each resource a mapping between process
|
|
instance and a unique ID.
|
|
* The `lastAssignments` attribute memorizes the last assigned ID for each
|
|
resource to prevent reassigning the same IDs, until the maximum ID limit has
|
|
been reached.
|
|
|
|
When updating the processes model, you can run the following command to update
|
|
the ID assignments:
|
|
|
|
```bash
|
|
$ nixproc-id-assign --id-resources idresources.nix --ids ids.nix --output-file ids.nix processes.nix
|
|
```
|
|
|
|
The difference between the above command invocation and the previous is that we
|
|
take our existing ID assignment in account -- for processes that were already
|
|
deployed previously we retain their ID assignments to prevent unnecessary
|
|
redeployments.
|
|
|
|
In addition to port numbers, we can also assign and retain unique UIDs and GIDs
|
|
per process instance. We can use a similar strategy to port numbers to propagate
|
|
these values as parameters, but a more convenient way is to instrument the
|
|
`createCredentials` function -- the above `processes.nix` expression propagates
|
|
the entire `ids` attribute set as a parameter to the constructors.
|
|
|
|
The constructors expression indirectly composes the `createCredentials` function
|
|
as follows:
|
|
|
|
```nix
|
|
{pkgs, ids ? {}, ...}:
|
|
|
|
{
|
|
createCredentials = import ../../create-credentials {
|
|
inherit (pkgs) stdenv;
|
|
inherit ids;
|
|
};
|
|
|
|
...
|
|
}
|
|
```
|
|
|
|
The `ids` attribute set is propagated to the function that composes the
|
|
`createCredentials` function. As a result, it will automatically assign the UIDs
|
|
and GIDs in the `ids.nix` expression when the user configures a user or group
|
|
with a name that exists in the `uids` and `gids` resource pools.
|
|
|
|
To make these UIDs and GIDs assignments go smoothly, it is recommended to give
|
|
a process the same process name, instance name, user and group names.
|
|
|
|
The `nixproc-id-assign` tool is basically just a wrapper around the
|
|
`dydisnix-id-assign` tool and internally converts a processes model to a Disnix
|
|
services model.
|
|
|
|
Writing integration tests
|
|
-------------------------
|
|
As explained in the introduction, the framework supports all kinds of
|
|
interesting features producing all kinds of variants of the same service, such
|
|
as multiple process managers, multiple process instances, unprivileged
|
|
deployments etc.
|
|
|
|
Although a service can support all these variants, writing a model does not
|
|
guarantee that it will always work under all circumstances. The Nix process
|
|
management framework supports code reuse, but does not facilitate a write once,
|
|
run anywhere approach.
|
|
|
|
To validate a service, we can use a test driver built on top of the NixOS test
|
|
driver that can be used to test multiple variants of a service.
|
|
|
|
The following Nix expression is an example of a test suite for the advanced
|
|
variant of the webapp example with two Nginx reverse proxies:
|
|
|
|
```nix
|
|
{ pkgs, testService, processManagers, profiles }:
|
|
|
|
testService {
|
|
inherit processManagers profiles;
|
|
|
|
exprFile = ./processes-advanced.nix;
|
|
|
|
readiness = {instanceName, instance, ...}:
|
|
''
|
|
machine.wait_for_open_port(${toString instance.port})
|
|
'';
|
|
|
|
tests = {instanceName, instance, ...}:
|
|
pkgs.lib.optionalString (instanceName == "nginx" || instanceName == "nginx2")
|
|
(pkgs.lib.concatMapStrings (webapp: ''
|
|
machine.succeed(
|
|
"curl --fail -H 'Host: ${webapp.dnsName}' http://localhost:${toString instance.port} | grep ': ${toString webapp.port}'"
|
|
)
|
|
'') instance.webapps);
|
|
|
|
}
|
|
```
|
|
|
|
The above Nix expression invokes `testService` with the following parameters:
|
|
* `processManagers` refers to a list of names of all the process managers that
|
|
should be tested.
|
|
* `profiles` refers to a list of configuration profiles that should be tested.
|
|
Currently, it supports `privileged` for privileged deployments, and
|
|
`unprivileged` for unprivileged deployments in an unprivileged user's home
|
|
directory, without changing user permissions.
|
|
* The `exprFile` parameter refers to a processes model of a system, such as
|
|
`processes-advanced.nix` capturing the properties of a system that consists
|
|
of multiple `webapp` and `nginx` instances, as described earlier.
|
|
* The `readiness` parameter refers to a function that does a readiness check
|
|
for each process instance. In the above example, it checks whether each service
|
|
is actually listening on the required TCP port.
|
|
* The `tests` parameter refers to a function that executes tests for each
|
|
process instance. In the above example, it ignores all but the `nginx`
|
|
instances, because explicitly testing a `webapp` instance is a redundant
|
|
operation. For each `nginx` instance, it checks whether all `webapp` instances
|
|
can be reached from it, by running the `curl` command.
|
|
|
|
The `readiness` and `tests` functions take `instanceName` as a parameter that
|
|
identifies the process instance in the processes model, and `instance` that
|
|
refers to the attribute set containing its configuration.
|
|
|
|
In the `readiness` and `tests` functions, it is also possible to refer to global
|
|
configuration parameters:
|
|
* `stateDir`. The directory in which state files are stored (typically `/var`
|
|
for privileged deployments)
|
|
* `runtimeDir`: The directory in which runtime files are stored (typically
|
|
`/var/run` for privileged installations).
|
|
* `forceDisableUserChange`. Indicates whether to disable user changes (for
|
|
unprivileged deployments) or not.
|
|
|
|
In addition to writing tests that work on instance level, it is also possible
|
|
to write tests on system level, with the following parameters (not shown in the
|
|
example):
|
|
|
|
* `initialTests`: instructions that run right after deploying the system, but
|
|
before the `readiness` checks, and instance-level `tests`.
|
|
* `postTests`: instructions that run after the instance-level `tests`.
|
|
|
|
The above functions also accept the same global configuration parameters, and
|
|
`processes` that refers to the entire processes model.
|
|
|
|
We can also configure other properties useful for testing:
|
|
* `systemPackages`: installs additional packages into the system profile of the
|
|
test virtual machine.
|
|
* `nixosConfig` defines a NixOS module with configuration properties that will
|
|
be added to the NixOS configuration of the test machine.
|
|
* `extraParams` propagates additional parameters to the processes model.
|
|
|
|
The Nix expression shown earlier is not self-contained -- it is a function
|
|
definition that needs to be invoked with all its required parameters including
|
|
the process managers and profiles that we want to test for.
|
|
|
|
We can compose tests in the following Nix expression:
|
|
|
|
```nix
|
|
{ nixpkgs ? <nixpkgs>
|
|
, system ? builtins.currentSystem
|
|
, processManagers ? [ "supervisord" "sysvinit" "systemd" "docker" "disnix" "s6-rc" ]
|
|
, profiles ? [ "privileged" "unprivileged" ]
|
|
}:
|
|
|
|
let
|
|
pkgs = import nixpkgs { inherit system; };
|
|
|
|
testService = import ../../nixproc/test-driver/universal.nix {
|
|
inherit nixpkgs system;
|
|
};
|
|
in
|
|
{
|
|
|
|
nginx-reverse-proxy-hostbased = import ./nginx-reverse-proxy-hostbased {
|
|
inherit pkgs processManagers profiles testService;
|
|
};
|
|
|
|
docker = import ./docker {
|
|
inherit pkgs processManagers profiles testService;
|
|
};
|
|
|
|
...
|
|
}
|
|
```
|
|
|
|
The above partial Nix expression (`default.nix`) invokes the function defined in
|
|
the previous Nix expression that resides in the `nginx-reverse-proxy-hostbased`
|
|
directory and propagates all required parameters. It also composes other test
|
|
cases, such as `docker`.
|
|
|
|
The parameters of the composition expression allow you to globally configure
|
|
the service variants:
|
|
|
|
* `processManagers` allows you to select the process managers you want to test
|
|
for.
|
|
* `profiles` allows you to select the configuration profiles.
|
|
|
|
With the following command, we can test our system as a privileged user, using
|
|
`systemd` as a process manager:
|
|
|
|
```bash
|
|
$ nix-build -A nginx-reverse-proxy-hostbased.privileged.systemd
|
|
```
|
|
|
|
we can also run the same test, but then as an unprivileged user:
|
|
|
|
```bash
|
|
$ nix-build -A nginx-reverse-proxy-hostbased.unprivileged.systemd
|
|
```
|
|
|
|
In addition to `systemd`, any configured process manager can be used that works
|
|
in NixOS. The following command runs a privileged test of the same service for
|
|
`sysvinit`:
|
|
|
|
```bash
|
|
$ nix-build -A nginx-reverse-proxy-hostbased.privileged.sysvinit
|
|
```
|
|
|
|
Although the test driver makes it possible to test all possible variants of a
|
|
service, doing so may be very expensive. In the above example, we have two
|
|
configuration profiles and six process managers, resulting in twelve possible
|
|
variants of the same service.
|
|
|
|
To get a reasonable level of confidence, it typically suffices to implement the
|
|
following strategy:
|
|
* Pick two process managers: one that prefers foreground processes
|
|
(e.g. `supervisord`) and one that prefers daemons (e.g. `sysvinit`).
|
|
This is the most significant difference (from a configuration perspective)
|
|
between all these different process managers.
|
|
* If a service supports multiple configuration variants, and multiple
|
|
instances, then create a processes model that concurrently deploys all
|
|
these variants.
|
|
|
|
Implementing the above strategy only requires you to test four variants,
|
|
providing a high degree of certainty that it will work with all other process
|
|
managers as well.
|
|
|
|
Since the test driver is built on top of the NixOS test driver (that is Linux
|
|
based), we cannot use the test driver to test service variants on different
|
|
operating systems. `launchd`, `bsdrc` and `cygrunsrv` can only be tested
|
|
manually for now.
|
|
|
|
Integration with Disnix
|
|
-----------------------
|
|
In addition to the fact that this toolset provides a `disnix` backend that
|
|
facilitates universal and easy local deployment, any process model is basically
|
|
a sub set of a Disnix services model.
|
|
|
|
By augmenting all processes in a processes model with a number of additional
|
|
properties, we can turn it into a fully functional Disnix services model:
|
|
|
|
```nix
|
|
{ pkgs ? import <nixpkgs> { inherit system; }
|
|
, system ? builtins.currentSystem
|
|
, distribution, invDistribution
|
|
, stateDir ? "/var"
|
|
, runtimeDir ? "${stateDir}/run"
|
|
, logDir ? "${stateDir}/log"
|
|
, tmpDir ? (if stateDir == "/var" then "/tmp" else "${stateDir}/tmp")
|
|
, forceDisableUserChange ? false
|
|
}:
|
|
|
|
let
|
|
processManager = "sysvinit";
|
|
|
|
constructors = import ./constructors.nix {
|
|
inherit pkgs stateDir runtimeDir logDir tmpDir forceDisableUserChange processManager;
|
|
};
|
|
in
|
|
rec {
|
|
webapp1 = rec {
|
|
name = "webapp1";
|
|
port = 5000;
|
|
dnsName = "webapp1.local";
|
|
|
|
pkg = constructors.webapp {
|
|
inherit port;
|
|
};
|
|
|
|
type = "sysvinit-script";
|
|
};
|
|
|
|
webapp2 = rec {
|
|
name = "webapp2";
|
|
port = 5001;
|
|
dnsName = "webapp2.local";
|
|
|
|
pkg = constructors.webapp {
|
|
inherit port;
|
|
};
|
|
|
|
type = "sysvinit-script";
|
|
};
|
|
|
|
nginx = rec {
|
|
name = "nginx";
|
|
port = 8080;
|
|
|
|
pkg = constructors.nginxReverseProxy {
|
|
webapps = [ webapp1 webapp2 ];
|
|
inherit port;
|
|
} {};
|
|
|
|
type = "sysvinit-script";
|
|
};
|
|
}
|
|
```
|
|
|
|
In the above Disnix services, the following changes were made:
|
|
|
|
* The `processManager` is hardcoded to `sysvinit`.
|
|
* Every process has been turned into a service by augmenting the following
|
|
properties: `name` corresponds to the key in attribute set, and `type`
|
|
to the Dysnomia plugin that manages its lifecycle. To manage the lifecycle
|
|
of a `sysvinit-script` we can use the Dysnomia plugin with the same name.
|
|
|
|
Dysnomia, the toolset that manages the lifecycles of services, has plugins for
|
|
the same process managers that this toolset supports. With a few small
|
|
modifications, we can make a universal services model that allows us to pick
|
|
any process management solution that this toolset supports based on the on the
|
|
value of the `processManager` parameter:
|
|
|
|
```nix
|
|
{ pkgs ? import <nixpkgs> { inherit system; }
|
|
, system ? builtins.currentSystem
|
|
, distribution, invDistribution
|
|
, stateDir ? "/var"
|
|
, runtimeDir ? "${stateDir}/run"
|
|
, logDir ? "${stateDir}/log"
|
|
, tmpDir ? (if stateDir == "/var" then "/tmp" else "${stateDir}/tmp")
|
|
, forceDisableUserChange ? false
|
|
, processManager ? "sysvinit"
|
|
}:
|
|
|
|
let
|
|
processType = import ../../nixproc/derive-dysnomia-process-type.nix {
|
|
inherit processManager;
|
|
};
|
|
|
|
constructors = import ./constructors.nix {
|
|
inherit pkgs stateDir runtimeDir logDir tmpDir forceDisableUserChange processManager;
|
|
};
|
|
in
|
|
rec {
|
|
webapp1 = rec {
|
|
name = "webapp1";
|
|
port = 5000;
|
|
dnsName = "webapp1.local";
|
|
|
|
pkg = constructors.webapp {
|
|
inherit port;
|
|
};
|
|
|
|
type = processType;
|
|
};
|
|
|
|
webapp2 = rec {
|
|
name = "webapp2";
|
|
port = 5001;
|
|
dnsName = "webapp2.local";
|
|
|
|
pkg = constructors.webapp {
|
|
inherit port;
|
|
};
|
|
|
|
type = processType;
|
|
};
|
|
|
|
nginx = rec {
|
|
name = "nginx";
|
|
port = 8080;
|
|
|
|
pkg = constructors.nginxReverseProxy {
|
|
webapps = [ webapp1 webapp2 ];
|
|
inherit port;
|
|
} {};
|
|
|
|
type = processType;
|
|
};
|
|
}
|
|
```
|
|
|
|
In the services model shown above, we have re-introduced the `processManager`
|
|
parameter. We use a convenience function that derives the `processType` from
|
|
the selected `processManager`. For example, `sysvinit` maps to `sysvinit-script`,
|
|
`systemd` to `systemd-unit`, `supervisord` to `supervisord-program` etc.
|
|
All the service's `type` attributes bind to the derived `processType`.
|
|
|
|
There is also a special case -- when `processManager` is `null`, then the
|
|
selected type will be `managed-process`, that works with process
|
|
manager-agnostic JSON configuration files that get converted to a
|
|
process-manager specific configuration on the target machines (with
|
|
the `nixproc-generate-config` tool) and deployed as such.
|
|
|
|
`managed-process` is useful when we want to deploy services in a network of
|
|
machines running various opeating and process managers by using the same
|
|
deployment specifications.
|
|
|
|
By combining the services model shown above with an infrastructure model, and
|
|
distribution model, we can deploy the system to a network of machines:
|
|
|
|
```bash
|
|
$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix
|
|
```
|
|
|
|
the following command allows us to pick a different process manager, such as
|
|
`systemd`:
|
|
|
|
```bash
|
|
$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix --extra-params '{ processManager = "systemd"; }'
|
|
```
|
|
|
|
Building a multi-process Docker container
|
|
-----------------------------------------
|
|
Another useful integration solution is generating multi-process Docker images.
|
|
We can build a Docker image that launches multiple processes managed by a
|
|
process manager that is both compatible with Linux and Docker.
|
|
|
|
To construct such as an image, we can evaluate a Nix expression (e.g.
|
|
`default.nix`) that looks as follows:
|
|
|
|
```nix
|
|
{ pkgs ? import <nixpkgs> { inherit system; }
|
|
, system ? builtins.currentSystem
|
|
}:
|
|
|
|
let
|
|
createMultiProcessImage = import ../../nixproc/create-image-from-steps/create-multi-process-image-universal.nix {
|
|
inherit pkgs;
|
|
};
|
|
in
|
|
createMultiProcessImage {
|
|
name = "multiprocess";
|
|
tag = "test";
|
|
|
|
exprFile = ../webapps-agnostic/processes.nix;
|
|
processManager = "supervisord"; # sysvinit, disnix, s6-rc are also valid options
|
|
|
|
interactive = true; # the default option
|
|
manpages = false; # the default option
|
|
forceDisableUserChange = false; # the default option
|
|
}
|
|
```
|
|
|
|
In the above expression, we evaluate the `createMultiProcessImage` function
|
|
with the following parameters:
|
|
|
|
* The `name` refers to the name of the image, whereas `tag` refers to a Docker
|
|
image version tag.
|
|
* The `exprFile` refers to a processes expression file that declares running
|
|
process instances (as shown earlier)
|
|
* The `processManager` parameter allows you to pick a process manager.
|
|
Currently, all the options shown above are supported.
|
|
* We can also specify whether we want to use the container interactively
|
|
with the `interactive` parameter (which defaults to: `true`). When
|
|
this setting has been enabled, a `.bashrc` will be configured to make
|
|
the bash shell usable, and a number of additional packages will be installed
|
|
for file and process management operations.
|
|
* We can also optionally install `man` in the container so that you can access
|
|
manual pages. By default, it is disabled
|
|
* It is also possible to adjust the state settings in the processes model.
|
|
With `forceDisableUserChange` we can disable user creation and user
|
|
switching. It is also possible to control the other state variables, such
|
|
as `stateDir`.
|
|
|
|
The function shown above is basically a thin wrapper around the
|
|
`dockerTools.buildImage` function in Nixpkgs and accepts the same parameters,
|
|
with a number of process management parameters added to it.
|
|
|
|
The corresponding deployment procedure of an image is also similar to ordinary
|
|
single root process images. For example, to build the image you can run:
|
|
|
|
```bash
|
|
$ nix-build
|
|
```
|
|
|
|
and load it into Docker as follows:
|
|
|
|
```bash
|
|
$ docker load -i result
|
|
```
|
|
|
|
We can deploy a container instance from the image in interactive mode as
|
|
follows:
|
|
|
|
```bash
|
|
$ docker run --name mycontainer --rm --network host -it multiprocess:test
|
|
```
|
|
|
|
When interactive mode has been enabled, you should be able to "connect" to
|
|
a container in which you can execute shell commands, for example to control
|
|
the life-cycle of the sub processes:
|
|
|
|
```bash
|
|
$ docker exec -it mycontainer /bin/bash
|
|
$ ps aux
|
|
```
|
|
|
|
Building a mutable multi-process Docker container
|
|
-------------------------------------------------
|
|
A big drawback of the multi-process container deployed in the previous
|
|
section is that its deployment is *immutable* -- the deployment is done from an
|
|
image that configures all process instances, but when it is desired to change
|
|
the configuration, a new image needs to be generated and the container must
|
|
be discarded and redeployed from that new image.
|
|
|
|
It is also possible to deploy *mutable* multi-process containers in which the
|
|
configuration of the managed system can be updated without the need to bring
|
|
the container down.
|
|
|
|
The following Nix expression (`default.nix`) can be used to construct a mutable
|
|
multi-process image:
|
|
|
|
```nix
|
|
{ pkgs ? import <nixpkgs> { inherit system; }
|
|
, system ? builtins.currentSystem
|
|
}:
|
|
|
|
let
|
|
createMutableMultiProcessImage = import ../../nixproc/create-image-from-steps/create-mutable-multi-process-image-universal.nix {
|
|
inherit pkgs;
|
|
};
|
|
in
|
|
createMutableMultiProcessImage {
|
|
name = "multiprocess";
|
|
tag = "test";
|
|
|
|
exprFile = ./processes.nix;
|
|
idResourcesFile = ./idresources.nix;
|
|
idsFile = ./ids.nix;
|
|
processManager = "supervisord";
|
|
|
|
interactive = true;
|
|
manpages = false;
|
|
forceDisableUserChange = false;
|
|
bootstrap = true;
|
|
}
|
|
```
|
|
|
|
The Nix expression shown above invokes the `createMutableMultiProcessImage`
|
|
function that has a similar interface to the immutable variant shown in the
|
|
previous section, with the following differences:
|
|
|
|
* The `exprFile` is also used for specifying the process model to deploy from,
|
|
but a notable difference is that for mutable containers this model is copied
|
|
into the container and deployed from within the container. If the `exprFile`
|
|
parameter is omitted, an empty configuration is deployed.
|
|
* To make deploying process models model possible that also use
|
|
`nixproc-id-assign` to automatically assign unique numeric IDs, the
|
|
`idResourcesFile` and `idsFile` parameters can be used to copy these models
|
|
into the container as well. These parameters are not mandatory.
|
|
* As a container entry point, a *bootstrap* script is executed, that on first
|
|
deployment, uses the Nix package manager, and the corresponding
|
|
`nixproc-*-switch` tool to deploy the system.
|
|
* The `bootstrap` parameter allows you to disable the bootstrap entry point.
|
|
By default, it is enabled.
|
|
|
|
To make deployments in mutable containers possible, the processes model should
|
|
not contain any references to files on the local file system (with the exception
|
|
of the `ids.nix` model that should reside in the same base directory).
|
|
|
|
The following process model (`processes.nix`) eliminates local file dependencies
|
|
by using `builtins.fetchGit` to obtain the Nix process management framework:
|
|
|
|
```nix
|
|
{ pkgs ? import <nixpkgs> { inherit system; }
|
|
, system ? builtins.currentSystem
|
|
, stateDir ? "/var"
|
|
, runtimeDir ? "${stateDir}/run"
|
|
, logDir ? "${stateDir}/log"
|
|
, cacheDir ? "${stateDir}/cache"
|
|
, tmpDir ? (if stateDir == "/var" then "/tmp" else "${stateDir}/tmp")
|
|
, forceDisableUserChange ? false
|
|
, processManager
|
|
, webappMode ? null
|
|
}:
|
|
|
|
let
|
|
nix-processmgmt = builtins.fetchGit {
|
|
url = https://github.com/svanderburg/nix-processmgmt.git;
|
|
ref = "master";
|
|
};
|
|
|
|
ids = if builtins.pathExists ./ids.nix then (import ./ids.nix).ids else {};
|
|
|
|
sharedConstructors = import "${nix-processmgmt}/examples/services-agnostic/constructors.nix" {
|
|
inherit pkgs stateDir runtimeDir logDir cacheDir tmpDir forceDisableUserChange processManager ids;
|
|
};
|
|
|
|
constructors = import "${nix-processmgmt}/examples/webapps-agnostic/constructors.nix" {
|
|
inherit pkgs stateDir runtimeDir logDir tmpDir forceDisableUserChange processManager webappMode ids;
|
|
};
|
|
in
|
|
rec {
|
|
webapp = rec {
|
|
port = ids.webappPorts.webapp or 0;
|
|
dnsName = "webapp.local";
|
|
|
|
pkg = constructors.webapp {
|
|
inherit port;
|
|
};
|
|
|
|
requiresUniqueIdsFor = [ "webappPorts" "uids" "gids" ];
|
|
};
|
|
|
|
nginx = rec {
|
|
port = ids.nginxPorts.nginx or 0;
|
|
|
|
pkg = sharedConstructors.nginxReverseProxyHostBased {
|
|
webapps = [ webapp ];
|
|
inherit port;
|
|
} {};
|
|
|
|
requiresUniqueIdsFor = [ "nginxPorts" "uids" "gids" ];
|
|
};
|
|
}
|
|
```
|
|
|
|
To deploy a mutable multi-process image container, we can run the following
|
|
command to build the image:
|
|
|
|
```bash
|
|
$ nix-build
|
|
```
|
|
|
|
and load it into Docker as follows:
|
|
|
|
```bash
|
|
$ docker load -i result
|
|
```
|
|
|
|
We can deploy a container instance from the image in interactive mode as
|
|
follows:
|
|
|
|
```bash
|
|
$ docker run --name mycontainer --rm --network host -it multiprocess:test
|
|
```
|
|
|
|
On first startup, the container will carry out a bootstrap procedure that uses
|
|
the Nix process management framework to deploy all the processes in the
|
|
processes model.
|
|
|
|
When the deployment of the system is complete, we can "connect" to the container
|
|
instance as follows:
|
|
|
|
```bash
|
|
$ docker exec -it mycontainer /bin/bash
|
|
```
|
|
|
|
In the container, we can edit the process model (to for example, add a new
|
|
`webapp` instance):
|
|
|
|
```bash
|
|
$ mcedit /etc/nixproc/processes.nix
|
|
```
|
|
|
|
and redeploy the new configuration with the following command:
|
|
|
|
```bash
|
|
$ nixproc-supervisord-switch
|
|
```
|
|
|
|
then the new configuration should become active without the need to restart the
|
|
container. Moreover, when stopping the container and starting it again, the last
|
|
deployed configuration should become active again.
|
|
|
|
Building a Nix-enabled Docker container
|
|
---------------------------------------
|
|
One of the interesting aspects of a mutable multi-process image is that it
|
|
provides a fully featured Nix installation in a container that can be used
|
|
to deploy arbitrary Nix packages.
|
|
|
|
It is also possible to generate an image whose only purpose is to provide a
|
|
working Nix installation:
|
|
|
|
```nix
|
|
{ pkgs ? import <nixpkgs> { inherit system; }
|
|
, system ? builtins.currentSystem
|
|
}:
|
|
|
|
let
|
|
createNixImage = import ../../nixproc/create-image-from-steps/create-nix-image.nix {
|
|
inherit pkgs;
|
|
};
|
|
in
|
|
createNixImage {
|
|
name = "foobar";
|
|
tag = "test";
|
|
}
|
|
```
|
|
|
|
Examples
|
|
========
|
|
This repository contains a number of example systems, that can be found in the
|
|
`examples/` folder:
|
|
|
|
* `webapps-sysvinit` is a processes configuration set using the example `webapp`
|
|
process described in this README, with `nginx` reverse proxies. The
|
|
configuration is specifically written to use sysvinit as a process manager.
|
|
* `webapps-agnostic` is the same as the previous example, but using a process
|
|
manager agnostic configuration. It can be used to target all process managers
|
|
that this toolset supports.
|
|
* `services-agnostic` is a process manager-agnostic configuration set of
|
|
additional system services used for tests, such as docker, supervisord, and
|
|
nginx
|
|
* `service-containers-agnostic` extends the previous examples with configuration
|
|
files so that these system services can be deployed as Disnix containers --
|
|
services in which other services can be hosted.
|
|
* `multi-process-image` is an example demonstrating how to construct a Docker
|
|
image that concurrently runs all processes described in the `webapps-agnostic`
|
|
example managed by a process management solution of choice.
|
|
|
|
The
|
|
[Nix process management services](https://github.com/svanderburg/nix-processmgmt-services)
|
|
repository contains a collection of commonly used services that can be managed
|
|
with the Nix process management framework.
|
|
|
|
Troubleshooting
|
|
===============
|
|
This section contains a number of known problems and their resolutions.
|
|
|
|
Inspecting log files
|
|
--------------------
|
|
When a service does not work as expected, then it is typically desired to check
|
|
the logs of the corresponding service. Although many process management concepts
|
|
are standardized by this framework, logging is not standardized at all.
|
|
|
|
This section contains some pointers for some of the process management solution
|
|
targets that are currently implemented.
|
|
|
|
### systemd and docker
|
|
|
|
Some process/service managers have their own logging facility. For example,
|
|
`systemd` provides `journalctl`, and `docker` provides `docker logs`. They
|
|
automatically capture the output (the `stdout` and `stderr`) of foreground
|
|
processes.
|
|
|
|
### sysvinit, bsdrc, disnix
|
|
|
|
For process management solutions that rely on processes that daemonize on their
|
|
own (`sysvinit`, `bsdrc` and `disnix`), you need to consult the logs that are
|
|
managed by the services themselves (a daemon typically detaches itself from the
|
|
`stdout` and `stderr`. As a result, a process manager cannot do it).
|
|
|
|
Services that only provide foreground processes are automatically daemonized
|
|
with the `daemon` command by these three backends. By default, the `daemon`
|
|
command will capture their outputs in log files with a `nixproc-` prefix in
|
|
the log directory. On a production system, such a log file could be:
|
|
`/var/log/nixproc-myservice.log` for services that are started as root users
|
|
and `/tmp/nixproc-myservice.log` for services that are started as unprivileged
|
|
users.
|
|
|
|
### supervisord
|
|
|
|
`supervisord` will (if no settings have been configured) store log files
|
|
(capturing the `stdout` and `stderr` of each process) in the temp directory
|
|
(typically `/tmp` or the value of the `TMPDIR` environment variable).
|
|
|
|
### cygrunsrv
|
|
|
|
By default, the `stderr` of `cygrunsrv` managed services are captured in the log
|
|
folder. An example could be: `/var/log/myservice.log`.
|
|
|
|
### s6-rc
|
|
|
|
If there is no logging configured, any output produced by supervised processes
|
|
is redirected to the `s6-svscan` process that supervises the entire service
|
|
dependency tree.
|
|
|
|
In this framework, by default, `longrun` services are automatically configured
|
|
in such a way that there is also logging companion service that captures its
|
|
output.
|
|
|
|
The output captured by the logging companion services are stored in the `s6-log`
|
|
sub folder in the log directory (which is typically
|
|
`/var/log/s6-log/<service name>` on most systems).
|
|
|
|
License
|
|
=======
|
|
The contents of this package is available under the same license as Nixpkgs --
|
|
the [MIT](https://opensource.org/licenses/MIT) license.
|