The load of the isolates

IsolateLoads answers one question: which isolates does the application have, and how busy is each? It gives one row per isolate, or per thread of a Flutter engine, with how late its event loop runs, the share of the time it works, and what it works on. With Spring Boot it is the bean isolateLoads.

Two things read it: the task manager of a Flutter app and the actuator's isolates endpoint. Your own code can too. The same measurements decide the scaling of the web server's isolates.

Why

"The server is slow" has two very different causes. Either an isolate computes and cannot take the next request, or it waits for a database and is free. The first is fixed by another isolate, the second is not. The rows tell which: a busy isolate runs late, a waiting one does not.

A compiled Dart program cannot list its isolates, and cannot ask how much processor time one uses. So the framework measures what can be measured from inside, and this page says where that falls short.

Example

import 'dart:isolate';

import 'package:spring_boot_dart/spring_boot_dart.dart';

@Service()
class Diagnostics {
  Diagnostics(this.loads);
  final IsolateLoads loads;

  // One line per row: "main (main): busy 0.02, lag 0:00:00.000180".
  List<String> rows() => [
    for (final load in loads.all())
      '${load.name} (${load.kind}): busy ${load.busy}, lag ${load.lag}',
  ];

  // An isolate the application starts itself is a row while it runs.
  Future<Report> build() => loads.track('report', Isolate.run(buildReport));
}

The first all() starts the measuring and has no lag yet. Read it again a moment later.

A row

A row is an IsolateLoad. Every value but name and kind is null when its source cannot measure it.

Field In JSON Meaning
name name The name the logs show: main, isolate-2, task-1, raster
kind kind What it is; see below
busy busy The share of the time it was working, 0 to 1
lag lagMicros How late the event loop ran a timer, on average: near zero when the isolate is free, the length of what blocks it when it is not
maxLag maxLagMicros The longest lag measured
active active What it is working on now: requests in flight, tasks running
memory memory Memory in bytes. Isolates share the memory of their process, so only the main row has it: the resident size of the process
details details What only one source measures, by name: requests, queued, tasks. A duration is in microseconds, under a name that ends in Micros

toJson() and IsolateLoad.fromJson use the JSON names. That is how a row crosses from one isolate to another, and what the actuator answers. A value that is null is left out.

kind The row is
main The application's first isolate
server An isolate of the web server's pool
task An isolate of the task executor
engine A thread of the Flutter engine
tracked Work shown with track
another word A source of your own

The sources

IsolateLoads measures nothing itself. It asks every IsolateLoadSource bean, in order:

Bean Package Rows
isolateLoadSampler spring_boot_dart The isolate it runs in: lag, maxLag, busy, and memory in the first isolate
isolateTaskExecutor spring_boot_dart One per isolate of the task executor that exists now
webServerLoadSource spring_boot_starter_web Adds to the isolate's own row: active (requests in flight) and the details requests (all it has taken) and refused (answered 503 because the server was full)
serverIsolatePool spring_boot_starter_web The rows of the other isolates of the server's pool, as each last sent them
flutterEngineLoads spring_boot_starter_flutter The engine's threads (raster, platform, io) and the frames' build times in main, with the Flutter task manager in the build

A source of your own is a bean that implements the interface, for isolates the application starts and manages itself:

@Component()
class EncoderLoads implements IsolateLoadSource {
  EncoderLoads(this.encoders);
  final EncoderPool encoders;

  @override
  List<IsolateLoad> loads() => [
    for (final encoder in encoders.all)
      IsolateLoad(
        name: encoder.name,
        kind: 'encoder',
        active: encoder.running,
        details: {'frames': encoder.frames},
      ),
  ];
}

loads() is called at every read, perhaps once a second. Answer from what you already know; do not wait for anything.

Rows of one name are one row

A source may add to a row another source gives, by giving a load of the same name. The web server does: the sampler gives main its lag, the server gives main its requests in flight, and the reader sees one row with both.

The merge takes, for each value, the first source that has it. So a later source cannot replace the busy of an earlier one, only fill what is missing. name and kind are the first source's. The details of both are kept; for a detail of the same name the later source wins.

track

No source sees an isolate started with Isolate.run or compute. track shows it:

final report = await loads.track('report', Isolate.run(buildReport));

While the future runs there is a row named report, of kind tracked, with busy: 1 and active: 1. Two tracked futures of one name are one row with active: 2. The row goes when the last of them completes, also with an error. track returns the future it was given.

It shows that the work runs, not how hard: nothing is measured inside that isolate. Work that fits the task executor gets real rows without track.

How lag and busy are measured

IsolateLoadSampler measures the isolate it runs in with one timer.

What that cannot tell:

A task executor's row has another busy: the share of the time during which the isolate had a task, over the last second or more. A task that awaits counts for all its time. It is worked out at most once a second, so several readers see the same value.

With a pool of server isolates

The rows of the other isolates are measured there and sent to the first isolate, which keeps the last answer of each.

Reports from every isolate

Numbers a bean keeps are kept once per isolate. An IsolateReporter bean makes them available for the whole application: each isolate is asked in its own isolate, and the answers are kept in the first isolate's IsolateReports, the bean isolateReports.

@Component()
class BasketCount implements IsolateReporter {
  BasketCount(this.baskets);
  final Baskets baskets;

  @override
  String get id => 'baskets';

  @override
  Object? report() => baskets.open; // of this isolate
}

// In the first isolate:
await reports.refresh();
final open = reports.of('baskets', own: 'main');
// {main: 12, isolate-1: 9, isolate-2: 14}
Member of IsolateReports Meaning
own() What this isolate's reporters tell now, by id
of(id, own: name) The reports named id by isolate: this isolate's first, under the name given, then the others in the order of their numbers (isolate-2 before isolate-10)
refresh() Asks the other isolates now. Completes at once without a pool
put, remove Called by the pool

The web starter's requests per second are built this way: each isolate counts, the first adds up.

In Spring (Java)

Spring has no counterpart: a JVM's threads share their beans, so a number kept in a bean is the application's. Nearest is the actuator's threaddump endpoint. Here each isolate measures itself and tells the first one.

See also