The task executor

IsolateTaskExecutor runs a function in another isolate and gives its result as a Future. The isolates come from a pool that starts them when tasks arrive and stops them when they are idle. With Spring Boot it is the bean isolateTaskExecutor.

Why

An isolate does one thing at a time. While a handler parses 20 MB of JSON, the server answers nobody else; while a screen resizes an image, the app draws no frame. await does not help: it frees the isolate only while it waits for something, not while it computes.

Work of that kind belongs in another isolate: parsing megabytes of JSON, hashing, resizing an image, building a report. The executor keeps the isolates ready, so a task does not pay for starting one each time, and limits how many run at once.

It is not the TaskExecutor behind @Async: that one runs a task in a later turn of the same isolate, which does not help work that computes (see @Async).

Example

import 'package:spring_boot_dart/spring_boot_dart.dart';

@Service()
class Reports {
  Reports(this.isolates);
  final IsolateTaskExecutor isolates;

  // The calling isolate goes on answering while another one parses.
  Future<Summary> summarize(String csv) =>
      isolates.submitWith(_summarize, csv);

  // Static: only the argument is copied to the other isolate.
  static Summary _summarize(String csv) => Summary.parse(csv);
}
# lib/resources/application.yml
spring:
  task:
    execution:
      isolates:
        core-size: 2          # not set: the processors less one
        queue-capacity: 100   # not set: no limit
        max-size: 4           # reached only when the queue is full
        keep-alive: 30s

No isolate is started before the first task.

submit and submitWith

Member Meaning
Future<R> submitWith<M, R>(task, message) Runs task(message) in an isolate of the pool. For a top-level or static function: only message is copied
Future<R> submit<R>(task) Runs a closure. Everything the closure uses from around it is copied with it
poolSize The isolates running now
activeCount The tasks running now
queueSize The tasks waiting for an isolate
Future<void> shutdown() Ends the pool. The context calls it when it closes

A task may return a value or a Future. An isolate runs one task at a time, also while the task awaits. A task has no timeout and cannot be cancelled: one that never ends keeps its isolate.

What crosses

An isolate shares no memory with another. The function, its argument and its result are copied.

// Copies the whole service, and fails if it holds a socket:
isolates.submit(() => _summarize(this.lastCsv));

// Copies one string:
isolates.submitWith(_summarize, lastCsv);

The pool

A task goes through these rules, in order:

  1. An isolate is free: it runs the task.
  2. Fewer than core-size isolates run: a new one is started for it.
  3. The queue has room (queue-capacity; not set: always): the task waits.
  4. Fewer than max-size isolates run: a new one is started for it.
  5. Otherwise the task is refused with an IsolateTaskRejectedException.

Consequences:

IsolateTaskRejectedException: the pool has its 4 isolates busy and its queue is full (100); raise spring.task.execution.isolates.max-size or queue-capacity

The pool logs at debug, on the logger spring_core_dart.isolate.task:

Starting isolate task-1: a task found no free isolate (1 of 4)
Starting isolate task-3: the queue is full (100) (3 of 4)
Isolate task-1 idle for 0:00:30.000000: stopped (2 left)

In a browser

A browser has no isolates to start. The executor is there all the same, with the same API: the task runs in the calling isolate, in a later turn of its event loop.

The properties

Key Type Default Meaning
spring.task.execution.isolates.core-size int not set: one less than the machine has processors, at least one The isolates started before tasks wait in the queue
spring.task.execution.isolates.max-size int not set: as many as core-size The most isolates, reached only when the queue is full
spring.task.execution.isolates.queue-capacity int not set: any number How many tasks may wait for an isolate
spring.task.execution.isolates.keep-alive Duration 60s How long an idle isolate stays before it is stopped
spring.task.execution.isolates.allow-core-timeout bool true Whether the first core-size isolates are stopped when idle too
spring.task.execution.isolates.name-prefix String task- The isolates' names in the logs, a number added: task-1

Without Spring Boot, IsolateTaskExecutor(settings: IsolateTaskExecutorSettings(coreSize: 2, queueCapacity: 100, maxSize: 4)) takes the same values; call shutdown() yourself.

With a pool of server isolates

Each isolate of the web server's pool runs the whole application, so each has an executor of its own, with its own core-size. Its isolates carry the name of their server isolate: isolate-2-task-1. With server.isolates.count: 4 and core-size: 2, up to twelve isolates compute at once on the machine: four that serve and eight that run tasks.

In the task manager

Each isolate of the pool is a row of the task manager, of kind task: busy is the share of the time it spent in tasks over the last second or more, active is 1 while it runs one, and the details have tasks (how many it has finished) and, on the first row, queued (how many wait).

In Spring (Java)

Future<Summary> summary = taskExecutor.submit(() -> Summary.parse(csv));
spring.task.execution.pool.core-size=2
spring.task.execution.pool.max-size=4
spring.task.execution.pool.queue-capacity=100
spring.task.execution.pool.keep-alive=30s

IsolateTaskExecutor is ThreadPoolTaskExecutor.submit(callable) with the pool of spring.task.execution.pool.*, and IsolateTaskRejectedException is TaskRejectedException. The difference is the one that matters everywhere here: a thread shares the heap, an isolate does not. A Java task reads the beans around it; a task here gets a copy of what it is given, and nothing else.

See also