The task executor
IsolateTaskExecutor runs a function in another isolate and gives its result as a Future. The isolates come from a pool that starts them when tasks arrive and stops them when they are idle. With Spring Boot it is the bean isolateTaskExecutor.
Why
An isolate does one thing at a time. While a handler parses 20 MB of JSON, the server answers nobody else; while a screen resizes an image, the app draws no frame. await does not help: it frees the isolate only while it waits for something, not while it computes.
Work of that kind belongs in another isolate: parsing megabytes of JSON, hashing, resizing an image, building a report. The executor keeps the isolates ready, so a task does not pay for starting one each time, and limits how many run at once.
It is not the TaskExecutor behind @Async: that one runs a task in a later turn of the same isolate, which does not help work that computes (see @Async).
Example
import 'package:spring_boot_dart/spring_boot_dart.dart';
@Service()
class Reports {
Reports(this.isolates);
final IsolateTaskExecutor isolates;
// The calling isolate goes on answering while another one parses.
Future<Summary> summarize(String csv) =>
isolates.submitWith(_summarize, csv);
// Static: only the argument is copied to the other isolate.
static Summary _summarize(String csv) => Summary.parse(csv);
}
# lib/resources/application.yml
spring:
task:
execution:
isolates:
core-size: 2 # not set: the processors less one
queue-capacity: 100 # not set: no limit
max-size: 4 # reached only when the queue is full
keep-alive: 30s
No isolate is started before the first task.
submit and submitWith
| Member | Meaning |
|---|---|
Future<R> submitWith<M, R>(task, message) |
Runs task(message) in an isolate of the pool. For a top-level or static function: only message is copied |
Future<R> submit<R>(task) |
Runs a closure. Everything the closure uses from around it is copied with it |
poolSize |
The isolates running now |
activeCount |
The tasks running now |
queueSize |
The tasks waiting for an isolate |
Future<void> shutdown() |
Ends the pool. The context calls it when it closes |
A task may return a value or a Future. An isolate runs one task at a time, also while the task awaits. A task has no timeout and cannot be cancelled: one that never ends keeps its isolate.
What crosses
An isolate shares no memory with another. The function, its argument and its result are copied.
- Prefer
submitWithwith a top-level or static function and the data as its argument. Then you see what is copied: the argument. - A closure copies what it uses.
submit(() => parse(text))copiestext. A closure that reads a field copies the whole object the field belongs to, and everything that object reaches. - Some objects cannot be copied: a socket, a timer, a
ReceivePort. A closure with one of them in reach fails the call with anArgumentError. The pool goes on; the next task runs. - The beans do not exist there. A task isolate does not run the application: no context, no
@Autowired, no configuration. Give the task what it needs as its argument. - Nor does the logging configuration. What a task logs is written in the default format under the isolate's name:
[task-1] [neon.reports] INFO: 1200 rows. - An error comes back. What the task throws is the error of the future, with its type (
FormatException), and the isolate goes on with the next task. - An error or a result that cannot be copied back is an
IsolateTaskExceptioninstead: the text of the error and the stack trace of the task's isolate (message,remoteStackTrace).
// Copies the whole service, and fails if it holds a socket:
isolates.submit(() => _summarize(this.lastCsv));
// Copies one string:
isolates.submitWith(_summarize, lastCsv);
The pool
A task goes through these rules, in order:
- An isolate is free: it runs the task.
- Fewer than
core-sizeisolates run: a new one is started for it. - The queue has room (
queue-capacity; not set: always): the task waits. - Fewer than
max-sizeisolates run: a new one is started for it. - Otherwise the task is refused with an
IsolateTaskRejectedException.
Consequences:
- Without
queue-capacitythe pool never grows pastcore-size. Tasks wait, however many.max-sizehas an effect only with aqueue-capacity. - The refusal is thrown by the call, not given as the future's error:
submititself throws. Its message names the keys to raise:
IsolateTaskRejectedException: the pool has its 4 isolates busy and its queue is full (100); raise spring.task.execution.isolates.max-size or queue-capacity
- An idle isolate leaves after
keep-alive. Withallow-core-timeout: true(the default) the firstcore-sizeisolates leave too, so an application that submits nothing has no task isolates. The next task starts one again, in a few milliseconds. - An isolate that ends by itself (killed, or an error outside a task) fails its task with an
IsolateTaskException(the isolate task-1 ended: ...) and is logged as a warning,Isolate task-1 failed. The queue goes on with a new isolate. - At shutdown waiting tasks and running ones fail with an
IsolateTaskRejectedException(the executor was shut down), the isolates stop, and a latersubmitthrows the same.
The pool logs at debug, on the logger spring_core_dart.isolate.task:
Starting isolate task-1: a task found no free isolate (1 of 4)
Starting isolate task-3: the queue is full (100) (3 of 4)
Isolate task-1 idle for 0:00:30.000000: stopped (2 left)
In a browser
A browser has no isolates to start. The executor is there all the same, with the same API: the task runs in the calling isolate, in a later turn of its event loop.
- Code that calls
submitcompiles and runs on every platform, and the browser build has no isolate code. - A long task still blocks the page. It only starts later; it is not moved away.
- Nothing is copied, so a closure may use anything. Code that works in a browser can still fail on a phone, where the closure is copied: test where the isolates are real.
poolSizeandqueueSizeare0, and the pool rules do not apply: nothing is refused but aftershutdown.
The properties
| Key | Type | Default | Meaning |
|---|---|---|---|
spring.task.execution.isolates.core-size |
int |
not set: one less than the machine has processors, at least one | The isolates started before tasks wait in the queue |
spring.task.execution.isolates.max-size |
int |
not set: as many as core-size |
The most isolates, reached only when the queue is full |
spring.task.execution.isolates.queue-capacity |
int |
not set: any number | How many tasks may wait for an isolate |
spring.task.execution.isolates.keep-alive |
Duration |
60s |
How long an idle isolate stays before it is stopped |
spring.task.execution.isolates.allow-core-timeout |
bool |
true |
Whether the first core-size isolates are stopped when idle too |
spring.task.execution.isolates.name-prefix |
String |
task- |
The isolates' names in the logs, a number added: task-1 |
Without Spring Boot, IsolateTaskExecutor(settings: IsolateTaskExecutorSettings(coreSize: 2, queueCapacity: 100, maxSize: 4)) takes the same values; call shutdown() yourself.
With a pool of server isolates
Each isolate of the web server's pool runs the whole application, so each has an executor of its own, with its own core-size. Its isolates carry the name of their server isolate: isolate-2-task-1. With server.isolates.count: 4 and core-size: 2, up to twelve isolates compute at once on the machine: four that serve and eight that run tasks.
In the task manager
Each isolate of the pool is a row of the task manager, of kind task: busy is the share of the time it spent in tasks over the last second or more, active is 1 while it runs one, and the details have tasks (how many it has finished) and, on the first row, queued (how many wait).
In Spring (Java)
Future<Summary> summary = taskExecutor.submit(() -> Summary.parse(csv));
spring.task.execution.pool.core-size=2
spring.task.execution.pool.max-size=4
spring.task.execution.pool.queue-capacity=100
spring.task.execution.pool.keep-alive=30s
IsolateTaskExecutor is ThreadPoolTaskExecutor.submit(callable) with the pool of spring.task.execution.pool.*, and IsolateTaskRejectedException is TaskRejectedException. The difference is the one that matters everywhere here: a thread shares the heap, an isolate does not. A Java task reads the beans around it; a task here gets a copy of what it is given, and nothing else.