Requests per second
The debug feature qps counts the requests the server answers and works out, every few seconds, the requests per second, how long they took, how many are in flight and how many failed: for each isolate and for all together. It logs one line per interval, and the actuator's qps endpoint answers the same numbers as JSON, from whichever isolate takes the request.
It is in the build only with spring.web.debug.qps.enabled: true. Without that line the application has no code that counts its requests.
Why
"How much does the server take?" is the first question of a load test and of an evening when the server is slow. This answers it in one line, while it happens, and per isolate, so a pool that serves unevenly shows.
It is a debug feature, not a metrics system. It keeps one window, the last, and nothing over time.
Example
# lib/resources/application.yml
spring:
web:
debug:
qps:
enabled: true # read by the build
log-interval: 10s
dart run build_runner build # the property is read here
dart run bin/main.dart
Under load, the first isolate logs on spring.web.qps, at info:
1250.4 requests/s (p50 1.1 ms, p99 9.2 ms), 14 in flight, 0 errors; main 412.0, isolate-1 420.2, isolate-2 418.2
With one isolate the line ends after the errors. In an interval without a request, and with none in flight, nothing is logged.
The build decides
spring.web.debug.qps.enabled is a build property: the generator reads it when build_runner runs and puts the feature's module into the application, or does not.
spring.web.debug.qps.enabled |
The application |
|---|---|
true |
Has the filter that counts and RequestRates, in every build |
false, or not set |
Has none of that code |
- Set it in
application.ymlorapplication.properties, without a profile: the build does not read profile files, environment variables or--dart-definevalues. - Run
build_runneragain after changing it. - A property at run time cannot switch it on or off. The module has no runtime condition: what the build put in, runs.
- The module is native only, like the web starter itself. A browser build has none of it.
What is counted
RequestRateFilter, the bean requestRateFilter, is the first filter of the chain, around everything else: the security filter chain, the handler, the writing of the answer. For each request it counts:
| Count | Meaning |
|---|---|
| requests | The requests answered |
| errors | Those answered with a 5xx status, or that ended with an error (an error that says its status counts by it, any other as a 500) |
| in flight | The requests being answered now |
| latency | The time from the filter's start to its end, in buckets |
What that leaves out:
- The time a request waited for the isolate. The filter starts when the isolate gets to the request. A request that waited 200 ms behind others and was answered in 2 ms counts as 2 ms. The wait shows as lag, not here.
- Requests refused because the server was full. Those are answered before any filter (Overload). They are the
refusedof the task manager. - The network. The time is the server's, not the client's.
Percentiles
The latency of each request falls into a bucket. The upper bounds of the buckets start at 100 µs, and each is half as much again as the one before, up to about a minute; a last bucket takes what is slower.
A percentile is the upper bound of the bucket its request fell in. So p99 9.2 ms means: 99 % of the requests were answered within 9.2 ms, and the real figure is somewhere between that bound and the one before (two thirds of it). That is enough to tell 2 ms from 20 ms from 200 ms. It is not enough to compare 8 ms with 9 ms.
The total's percentiles are worked out from the buckets of all isolates added up, not averaged from the isolates' percentiles.
The window
RequestRates, the bean requestRates, lives in the application's first isolate. Every spring.web.debug.qps.log-interval it asks the isolates for their counts and keeps the difference to the last look: that is the window.
- Requests per second are the requests of the window over its length.
- A request counts in the window in which it was answered.
- An isolate that was started again counts from zero; its window is what it has now.
- An isolate added or removed by scaling appears in or leaves the line.
in flightis the number at the end of the window.
In code, the window is asked for like any answer of the first isolate, so the same code works in every isolate:
@RestController()
class LoadController {
LoadController(this.primary);
final PrimaryIsolate primary;
// The last window as JSON: what /actuator/qps answers. null before the first.
@GetMapping('/internal/load')
Future<Object?> load() => primary.ask(RequestRates.handlerId);
// Ends the window now: what /actuator/qps/now answers.
@GetMapping('/internal/load/now')
Future<Object?> now() => primary.ask(RequestRates.handlerId, 'now');
}
RequestRates.handlerId is web.qps. See Asking the first isolate.
A bean of the first isolate can take RequestRates itself and read the typed values:
@Component()
@ConditionalOnPrimaryIsolate()
class RateWatch {
RateWatch(this.rates);
final RequestRates rates;
@Scheduled(fixedRate: Duration(minutes: 1))
void check() {
final p99 = rates.last?.total.percentile(0.99);
if (p99 != null && p99 > const Duration(seconds: 1)) {
alerts.slow(p99);
}
}
}
| Member | Meaning |
|---|---|
RequestRates.last |
The last window, a RequestRateWindow; null before the first interval has passed |
RequestRates.look() |
Ends the window now, logs it and answers it |
RequestRateWindow.total |
All isolates together, a RequestRate |
RequestRateWindow.isolates |
Each isolate by name, the first isolate first |
RequestRate.qps, requests, errors, active, seconds |
The numbers of the window |
RequestRate.percentile(share) |
The latency under which that share of the requests were answered (0.5, 0.99); null without requests |
The RequestRates bean exists in the first isolate only. A bean that takes it must be there only too (@ConditionalOnPrimaryIsolate()), or the other isolates do not start; a bean of every isolate asks through PrimaryIsolate.
Out of the logs
The line is logged at info. To keep the counting and the endpoint without the lines:
logging:
level:
spring.web.qps: warn
The properties
| Key | Type | Default | Meaning |
|---|---|---|---|
spring.web.debug.qps.enabled |
bool |
false |
Whether the application counts its requests. Read by the generator: without true the build has none of the feature's code |
spring.web.debug.qps.log-interval |
Duration |
10s |
The length of a window: how often the rate is worked out and logged. 0: nothing is logged, and a window is as long as the time between two looks (look(), /actuator/qps/now) |
In Spring (Java)
Spring's counterpart is Micrometer's http.server.requests timer, read through the actuator's metrics endpoint or a monitoring system. This is smaller on purpose: one window, one line, and no code in the build unless you ask for it.