Load and scaling

How many requests a second does a server built on the framework answer, and what happens past that? The repository has a tool that measures it, a bench server it measures, and a nightly job that runs both. This page says what the numbers mean and how to get your own.

Why

A limit you have not measured is a guess. Two numbers describe a server under load:

Between the two the server is slow but alive; past the second it is not. Knowing both tells you how many isolates to run, where to set the limits that make a full server refuse quickly, and when to add machines.

What is measured

The bench server (tool/load_bench) has an endpoint per scenario, each a different cost:

Scenario What it costs
text The server alone: routing, a short answer
json, json-post The ObjectMapper: writing fifty objects, reading and validating one
secure The security filter chain with a bearer token on every request
db-read, db-write A repository on SQLite
static A static resource
download-*, upload-* Files of a few kilobytes, megabytes and hundreds of megabytes, streamed; uploads as multipart and as a raw body

Small requests are loaded in steps of a rate (requests a second), each step judged sustained or not. Large transfers are loaded in steps of concurrent connections and judged by megabytes a second and by the server's memory, which must stay flat: a server that keeps a file in memory fails with the first large one.

Beyond the sweeps:

Latency that tells the truth

The load comes from oha, at a fixed rate, with its latency correction on. Without it a load generator waits for a slow answer before it sends the next request: the server falls behind, the generator politely slows down with it, and the report shows good latencies for a server that was drowning (the "coordinated omission" problem). With the correction a request is timed from when it should have been sent.

Run it

brew install oha                      # or: cargo install oha --locked
git clone https://github.com/farum-azula/spring_core_dart
cd spring_core_dart && dart pub get

dart run tool/load.dart --scenarios text,json        # two scenarios
dart run tool/load.dart --scenarios api --isolates 1,2,4
dart run tool/load.dart --scenarios scaling,cpu,overload

The tool builds the bench server to a native executable, starts it, runs the steps, stops it, and writes report.md and report.json. Any option with a dot in its name is a property of the server: --server.isolates.count=4. --compare puts several reports side by side.

Numbers depend on the machine, and on what else it does: compare runs on the same machine, and ratios rather than absolute values. A laptop that runs the load generator beside the server measures both.

The full manual is docs/load-testing.md in the repository.

What one laptop measured

One run on 10 November 2026: an Apple M5 with 10 cores, macOS, Dart 3.13, the bench server as a native executable, oha on two threads on the same machine, 50 connections, steps of five seconds. Short steps on a machine that also runs the load generator: read the ratios, not the absolute numbers.

Scenario Isolates Highest sustainable QPS Breaking point Peak memory
text 1 39,980 55,000/s 62 MB
text 2 74,919 100,000/s 64 MB
text 4 74,972 130,000/s 67 MB
json 1 4,499 8,500/s 64 MB
json 2 8,497 16,000/s 66 MB
json 4 11,495 21,000/s 90 MB

CPU-bound work, 25 ms of hashing a request at 20 requests a second, with a light request measured beside it:

Where the work runs The light request's p99 Most work a second
In the request's isolate 24.4 ms 44.8
In the task executor 0.26 ms 163

A full server, and slow clients:

Check What happened
server.max-concurrent-requests=8, 600 requests a second of 50 ms each 74 % answered 503 in 0.15 ms (p99 0.6 ms) with Retry-After; the accepted ones took 51 ms, as when the server is free
server.max-connections=16, 64 connections The connections over the limit got 503 and Connection: close, p99 0.8 ms
200 connections sending their headers a line every half second, server.connection-timeout=2s All dropped after 3.8 s; 3 MB of memory; other requests unaffected
A body that stalls 408 after 2 s
A body that trickles, a byte every half second Still open after 11 s: the timeout counts silence, and a trickle is not silent. Set server.request-timeout to bound it

Not measured: the automatic policy after its change, the transfers on several isolates, steps longer than five seconds. The database scenario (db-read, a repository on SQLite, 128 connections): 17,994 sustainable requests a second on one isolate, 37,987 on two, 59,974 on four (peaks 23,322, 45,091 and 64,099: 2.75 times one isolate on four). Four isolates took a quarter of the requests each. Writes do not scale, 12,000 a second on one isolate and on four: SQLite takes one writer.

The same scenarios on PostgreSQL 17 (in Docker on the same laptop, given three cores; a connection pool of eight per isolate):

Isolates Reads: sustainable QPS Reads: peak Writes: sustainable QPS Writes: peak
1 3,997 5,757 1,998 2,702
2 7,997 9,332 1,998 4,014
4 11,991 13,056 3,996 5,812

Writes scale here (2.15 times one isolate at the peak, where SQLite stays flat), and reads still grow at four. The absolute numbers are a fraction of SQLite's: SQLite is a library in the process, PostgreSQL a server reached over the network, here through the virtual machine Docker runs on macOS. At four isolates the server used about two cores of ten while PostgreSQL had three: the database was the limit, not the framework. On a real database server, the ratios are the part to keep.

Scaling under load (GET /json, about 5,300 requests a second an isolate, 7,900 asked for):

Step What happened
One isolate to three, by hand No dip; the new isolates were ready in 0.02 s and the p99 fell from 650 ms to 1.5 ms in the next two-second window
Three back to one 3 of about 13,300 requests lost: a request already on its way on a keep-alive connection the leaving isolate had just closed. A client that sends again gets its answer
scaling: auto, 200 connections Up 3.8 s after the overload began, down 9.6 s after it ended, one step each way
scaling: auto, 50 connections Did not scale: the lag stayed at the threshold of that version (10 ms). The default is 5 ms since, and the timers' own lateness is taken off; not measured again

Transfers, one isolate: a 512 MB download at 3.6 GB a second with 75 MB of memory, the same as for 10 MB. Uploads lift the memory by about 220 MB whatever their size (10 MB or 512 MB), and it does not come back during the run. A multipart upload is about six times slower than the same file as a raw body (304 against 1,892 MB a second).

See also