Load and scaling
How many requests a second does a server built on the framework answer, and what happens past that? The repository has a tool that measures it, a bench server it measures, and a nightly job that runs both. This page says what the numbers mean and how to get your own.
Why
A limit you have not measured is a guess. Two numbers describe a server under load:
- the highest sustainable QPS: the most requests a second it answers in time (a p99 latency under a threshold, 100 ms by default) with next to no errors;
- the breaking point: the load at which it stops keeping up (timeouts, refused connections, a p99 of seconds), and whether it comes back when the load goes.
Between the two the server is slow but alive; past the second it is not. Knowing both tells you how many isolates to run, where to set the limits that make a full server refuse quickly, and when to add machines.
What is measured
The bench server (tool/load_bench) has an endpoint per scenario, each a different cost:
| Scenario | What it costs |
|---|---|
text |
The server alone: routing, a short answer |
json, json-post |
The ObjectMapper: writing fifty objects, reading and validating one |
secure |
The security filter chain with a bearer token on every request |
db-read, db-write |
A repository on SQLite |
static |
A static resource |
download-*, upload-* |
Files of a few kilobytes, megabytes and hundreds of megabytes, streamed; uploads as multipart and as a raw body |
Small requests are loaded in steps of a rate (requests a second), each step judged sustained or not. Large transfers are loaded in steps of concurrent connections and judged by megabytes a second and by the server's memory, which must stay flat: a server that keeps a file in memory fails with the first large one.
Beyond the sweeps:
- isolates: the same scenarios with one, two and four server isolates, and how much each added isolate brings;
- scaling while loaded: the isolates added and removed under a steady load (the dip, the time until a new isolate helps, errors while one drains), and the automatic policy under a ramp (does it scale in time, does it flap);
- CPU-bound work: a slow computation in the request's isolate against the same in the task executor, with a light request measured beside it;
- overload: what a full server answers and how fast (
server.max-concurrent-requests,server.max-connections), slow headers, a slow body, a body that is too large.
Latency that tells the truth
The load comes from oha, at a fixed rate, with its latency correction on. Without it a load generator waits for a slow answer before it sends the next request: the server falls behind, the generator politely slows down with it, and the report shows good latencies for a server that was drowning (the "coordinated omission" problem). With the correction a request is timed from when it should have been sent.
Run it
brew install oha # or: cargo install oha --locked
git clone https://github.com/farum-azula/spring_core_dart
cd spring_core_dart && dart pub get
dart run tool/load.dart --scenarios text,json # two scenarios
dart run tool/load.dart --scenarios api --isolates 1,2,4
dart run tool/load.dart --scenarios scaling,cpu,overload
The tool builds the bench server to a native executable, starts it, runs the steps, stops it, and writes report.md and report.json. Any option with a dot in its name is a property of the server: --server.isolates.count=4. --compare puts several reports side by side.
Numbers depend on the machine, and on what else it does: compare runs on the same machine, and ratios rather than absolute values. A laptop that runs the load generator beside the server measures both.
The full manual is docs/load-testing.md in the repository.
What one laptop measured
One run on 10 November 2026: an Apple M5 with 10 cores, macOS, Dart 3.13, the bench server as a native executable, oha on two threads on the same machine, 50 connections, steps of five seconds. Short steps on a machine that also runs the load generator: read the ratios, not the absolute numbers.
| Scenario | Isolates | Highest sustainable QPS | Breaking point | Peak memory |
|---|---|---|---|---|
text |
1 | 39,980 | 55,000/s | 62 MB |
text |
2 | 74,919 | 100,000/s | 64 MB |
text |
4 | 74,972 | 130,000/s | 67 MB |
json |
1 | 4,499 | 8,500/s | 64 MB |
json |
2 | 8,497 | 16,000/s | 66 MB |
json |
4 | 11,495 | 21,000/s | 90 MB |
- A second isolate nearly doubles both: 1.87 times for
text, 1.89 times forjson. - Four isolates bring 2.55 times one for
json, and no more than two fortext. At 75,000 requests a second the two threads of the load generator are a likely limit, and the steps were coarse; this run cannot tell the server's limit from the generator's. - Past its breaking point the server answered again within milliseconds of the load going, every time.
CPU-bound work, 25 ms of hashing a request at 20 requests a second, with a light request measured beside it:
| Where the work runs | The light request's p99 | Most work a second |
|---|---|---|
| In the request's isolate | 24.4 ms | 44.8 |
| In the task executor | 0.26 ms | 163 |
A full server, and slow clients:
| Check | What happened |
|---|---|
server.max-concurrent-requests=8, 600 requests a second of 50 ms each |
74 % answered 503 in 0.15 ms (p99 0.6 ms) with Retry-After; the accepted ones took 51 ms, as when the server is free |
server.max-connections=16, 64 connections |
The connections over the limit got 503 and Connection: close, p99 0.8 ms |
200 connections sending their headers a line every half second, server.connection-timeout=2s |
All dropped after 3.8 s; 3 MB of memory; other requests unaffected |
| A body that stalls | 408 after 2 s |
| A body that trickles, a byte every half second | Still open after 11 s: the timeout counts silence, and a trickle is not silent. Set server.request-timeout to bound it |
Not measured: the automatic policy after its change, the transfers on several isolates, steps longer than five seconds.
The database scenario (db-read, a repository on SQLite, 128 connections): 17,994 sustainable requests a second on one isolate, 37,987 on two, 59,974 on four (peaks 23,322, 45,091 and 64,099: 2.75 times one isolate on four). Four isolates took a quarter of the requests each. Writes do not scale, 12,000 a second on one isolate and on four: SQLite takes one writer.
The same scenarios on PostgreSQL 17 (in Docker on the same laptop, given three cores; a connection pool of eight per isolate):
| Isolates | Reads: sustainable QPS | Reads: peak | Writes: sustainable QPS | Writes: peak |
|---|---|---|---|---|
| 1 | 3,997 | 5,757 | 1,998 | 2,702 |
| 2 | 7,997 | 9,332 | 1,998 | 4,014 |
| 4 | 11,991 | 13,056 | 3,996 | 5,812 |
Writes scale here (2.15 times one isolate at the peak, where SQLite stays flat), and reads still grow at four. The absolute numbers are a fraction of SQLite's: SQLite is a library in the process, PostgreSQL a server reached over the network, here through the virtual machine Docker runs on macOS. At four isolates the server used about two cores of ten while PostgreSQL had three: the database was the limit, not the framework. On a real database server, the ratios are the part to keep.
Scaling under load (GET /json, about 5,300 requests a second an isolate, 7,900 asked for):
| Step | What happened |
|---|---|
| One isolate to three, by hand | No dip; the new isolates were ready in 0.02 s and the p99 fell from 650 ms to 1.5 ms in the next two-second window |
| Three back to one | 3 of about 13,300 requests lost: a request already on its way on a keep-alive connection the leaving isolate had just closed. A client that sends again gets its answer |
scaling: auto, 200 connections |
Up 3.8 s after the overload began, down 9.6 s after it ended, one step each way |
scaling: auto, 50 connections |
Did not scale: the lag stayed at the threshold of that version (10 ms). The default is 5 ms since, and the timers' own lateness is taken off; not measured again |
Transfers, one isolate: a 512 MB download at 3.6 GB a second with 75 MB of memory, the same as for 10 MB. Uploads lift the memory by about 220 MB whatever their size (10 MB or 512 MB), and it does not come back during the run. A multipart upload is about six times slower than the same file as a raw body (304 against 1,892 MB a second).