242 lines
13 KiB
HTML
242 lines
13 KiB
HTML
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<title>minimart-benchmark results</title>
|
|
<base href="http://localhost:4000/">
|
|
<style>
|
|
.logscale { background-color: #cfc; }
|
|
.linearscale { background-color: #ccf; }
|
|
.internallatency { background-color: #fcc; }
|
|
.serverstats { background-color: #ffc; }
|
|
.routingevents { background-color: #fcf; }
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<h1>Latest minimart-benchmark results</h1>
|
|
<h2>External latency</h2>
|
|
<p>
|
|
Measures round-trip latency of back-to-back requests at
|
|
varying numbers of simultaneously-open connections. The client
|
|
chooses an open connection at random for each successive ping.
|
|
</p>
|
|
<p>
|
|
The server variants:
|
|
</p>
|
|
<ul>
|
|
<li><em>prospect+tcp</em>:
|
|
Uses prospect (incremental NC), demand-matchers, and the prospect TCP driver actor.</li>
|
|
<li><em>minimart+tcp</em>:
|
|
Uses minimart (monolithic NC), demand-matchers, and the minimart TCP driver actor.</li>
|
|
<li><em>minimart</em>:
|
|
Uses minimart (monolithic NC) with Racket CML-style events used for TCP directly at ground level.</li>
|
|
<li><em>racket</em>:
|
|
The obvious Racket echo server using Racket threads and Racket I/O.</li>
|
|
<li><em>uv</em>:
|
|
A trivial C echo server written using <a href="https://github.com/libuv/libuv">libuv</a>.</li>
|
|
<li><em>erlang</em>:
|
|
A trivial echo server written using Erlang.</li>
|
|
</ul>
|
|
<p>
|
|
The server variants implemented in Racket collect server
|
|
statistics periodically as they run.
|
|
</p>
|
|
<div class="logscale">
|
|
<h3>Log-scale X axis</h3>
|
|
<table>
|
|
<tr>
|
|
<th>Server variant</th>
|
|
<th>Roundtrip latency</th>
|
|
<th>Seconds per additional connection</th>
|
|
<th>Roundtrips per second</th>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-prospect+tcp/--latest--">prospect+tcp</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-prospect+tcp/--latest--/client-ping-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-prospect+tcp/--latest--/client-grow-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-prospect+tcp/--latest--/client-ping-times/plot/0/2?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-minimart+tcp/--latest--">minimart+tcp</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart+tcp/--latest--/client-ping-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart+tcp/--latest--/client-grow-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart+tcp/--latest--/client-ping-times/plot/0/2?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-minimart/--latest--">minimart</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart/--latest--/client-ping-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart/--latest--/client-grow-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart/--latest--/client-ping-times/plot/0/2?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-racket/--latest--">racket</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-racket/--latest--/client-ping-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-racket/--latest--/client-grow-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-racket/--latest--/client-ping-times/plot/0/2?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-uv/--latest--">uv</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-uv/--latest--/client-ping-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-uv/--latest--/client-grow-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-uv/--latest--/client-ping-times/plot/0/2?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-erlang/--latest--">erlang</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-erlang/--latest--/client-ping-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-erlang/--latest--/client-grow-times/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-erlang/--latest--/client-ping-times/plot/0/2?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
</tr>
|
|
</table>
|
|
</div>
|
|
<div class="linearscale">
|
|
<h3>Linear-scale X axis</h3>
|
|
<table>
|
|
<tr>
|
|
<th>Server variant</th>
|
|
<th>Roundtrip latency</th>
|
|
<th>Seconds per additional connection</th>
|
|
<th>Roundtrips per second</th>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-prospect+tcp/--latest--">prospect+tcp</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-prospect+tcp/--latest--/client-ping-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-prospect+tcp/--latest--/client-grow-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-prospect+tcp/--latest--/client-ping-times/plot/0/2?x-max=30000&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-minimart+tcp/--latest--">minimart+tcp</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart+tcp/--latest--/client-ping-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart+tcp/--latest--/client-grow-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart+tcp/--latest--/client-ping-times/plot/0/2?x-max=30000&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-minimart/--latest--">minimart</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart/--latest--/client-ping-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart/--latest--/client-grow-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart/--latest--/client-ping-times/plot/0/2?x-max=30000&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-racket/--latest--">racket</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-racket/--latest--/client-ping-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-racket/--latest--/client-grow-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-racket/--latest--/client-ping-times/plot/0/2?x-max=30000&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-uv/--latest--">uv</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-uv/--latest--/client-ping-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-uv/--latest--/client-grow-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-uv/--latest--/client-ping-times/plot/0/2?x-max=30000&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-erlang/--latest--">erlang</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-erlang/--latest--/client-ping-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-erlang/--latest--/client-grow-times/plot/0/1?x-max=30000&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-erlang/--latest--/client-ping-times/plot/0/2?x-max=30000&y-min=0"></td>
|
|
</tr>
|
|
</table>
|
|
</div>
|
|
<div class="serverstats">
|
|
<h3>Server Statistics</h3>
|
|
<table>
|
|
<tr>
|
|
<th>Server variant</th>
|
|
<th>Server connections</th>
|
|
<th>Server memory use</th>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-prospect+tcp/--latest--">prospect+tcp</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-prospect+tcp/--latest--/server-connections/plot/0/1?y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-prospect+tcp/--latest--/server-memory-use/plot/0/1?y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-minimart+tcp/--latest--">minimart+tcp</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart+tcp/--latest--/server-connections/plot/0/1?y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart+tcp/--latest--/server-memory-use/plot/0/1?y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-minimart/--latest--">minimart</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart/--latest--/server-connections/plot/0/1?y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-minimart/--latest--/server-memory-use/plot/0/1?y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/external-latency-racket/--latest--">racket</a></th>
|
|
<td><img class="plot" src="log/minimart/external-latency-racket/--latest--/server-connections/plot/0/1?y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/external-latency-racket/--latest--/server-memory-use/plot/0/1?y-min=0"></td>
|
|
</tr>
|
|
</table>
|
|
</div>
|
|
<h2>Internal latency</h2>
|
|
<p>
|
|
Measures message delivery latency and actor boot time with
|
|
varying numbers of actors in a world.
|
|
</p>
|
|
<div class="internallatency">
|
|
<table>
|
|
<tr>
|
|
<th>Server variant</th>
|
|
<th>Single-message latency</th>
|
|
<th>Seconds per additional booted process</th>
|
|
<th>Messages per second</th>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/internal-latency/--latest--">minimart</a></th>
|
|
<td><img class="plot" src="log/minimart/internal-latency/--latest--/internal-latency/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/internal-latency/--latest--/internal-latency/plot/0/4?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/internal-latency/--latest--/internal-latency/plot/0/2?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/internal-latency-prospect/--latest--">prospect</a></th>
|
|
<td><img class="plot" src="log/minimart/internal-latency-prospect/--latest--/internal-latency/plot/0/1?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/internal-latency-prospect/--latest--/internal-latency/plot/0/4?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/internal-latency-prospect/--latest--/internal-latency/plot/0/2?logaxes=0&x-max=10.31&y-min=0"></td>
|
|
</tr>
|
|
</table>
|
|
</div>
|
|
<h2>Routing Events</h2>
|
|
<p>
|
|
Measures computation and dispatch time of routing events, plus
|
|
processing time of routing events in leaf actors.
|
|
</p>
|
|
<ul>
|
|
<li><em>minimart</em>:
|
|
Time taken to produce, convey, and process each routing event in varying-sized cliques in minimart (monolithic NC).</li>
|
|
<li><em>prospect</em>:
|
|
Time taken to produce, convey, and process each routing event in varying-sized cliques in prospect (incremental NC).</li>
|
|
</ul>
|
|
<div class="routingevents">
|
|
<table>
|
|
<tr>
|
|
<th>Variant</th>
|
|
<th>Production computation cost of each routing event</th>
|
|
<th>Production <em>and</em> consumption computation cost of each routing event</th>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/observe-all-minimart/--latest--">observe</a>/<a href="log/minimart/sum-all-minimart/--latest--">sum</a> minimart</a></th>
|
|
<td><img class="plot" src="log/minimart/observe-all-minimart/--latest--/presence-processing/plot/0/1?y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/sum-all-minimart/--latest--/presence-processing/plot/0/1?y-min=0"></td>
|
|
</tr>
|
|
<tr>
|
|
<th><a href="log/minimart/observe-all-prospect/--latest--">observe</a>/<a href="log/minimart/sum-all-prospect/--latest--">sum</a> prospect</a></th>
|
|
<td><img class="plot" src="log/minimart/observe-all-prospect/--latest--/presence-processing/plot/0/1?y-min=0"></td>
|
|
<td><img class="plot" src="log/minimart/sum-all-prospect/--latest--/presence-processing/plot/0/1?y-min=0"></td>
|
|
</tr>
|
|
</table>
|
|
</div>
|
|
<script>
|
|
(function () {
|
|
function reloadImage(e) {
|
|
this.src = this.src.replace(/#.*/, '') + '#' + (+(new Date()));
|
|
}
|
|
var scale = 0.5;
|
|
var imgs = document.getElementsByClassName('plot');
|
|
for (var i = 0; i < imgs.length; i++) {
|
|
var img = imgs[i];
|
|
if (img.tagName !== 'IMG') continue;
|
|
img.width = 700 * scale;
|
|
img.height = 450 * scale;
|
|
img.onclick = reloadImage;
|
|
}
|
|
})();
|
|
</script>
|
|
</body>
|
|
</html>
|