Requests per second

The debug feature qps counts the requests the server answers and works out, every few seconds, the requests per second, how long they took, how many are in flight and how many failed: for each isolate and for all together. It logs one line per interval, and the actuator's qps endpoint answers the same numbers as JSON, from whichever isolate takes the request.

It is in the build only with spring.web.debug.qps.enabled: true. Without that line the application has no code that counts its requests.

Why

"How much does the server take?" is the first question of a load test and of an evening when the server is slow. This answers it in one line, while it happens, and per isolate, so a pool that serves unevenly shows.

It is a debug feature, not a metrics system. It keeps one window, the last, and nothing over time.

Example

# lib/resources/application.yml
spring:
  web:
    debug:
      qps:
        enabled: true       # read by the build
        log-interval: 10s
dart run build_runner build   # the property is read here
dart run bin/main.dart

Under load, the first isolate logs on spring.web.qps, at info:

1250.4 requests/s (p50 1.1 ms, p99 9.2 ms), 14 in flight, 0 errors; main 412.0, isolate-1 420.2, isolate-2 418.2

With one isolate the line ends after the errors. In an interval without a request, and with none in flight, nothing is logged.

The build decides

spring.web.debug.qps.enabled is a build property: the generator reads it when build_runner runs and puts the feature's module into the application, or does not.

spring.web.debug.qps.enabled The application
true Has the filter that counts and RequestRates, in every build
false, or not set Has none of that code

What is counted

RequestRateFilter, the bean requestRateFilter, is the first filter of the chain, around everything else: the security filter chain, the handler, the writing of the answer. For each request it counts:

Count Meaning
requests The requests answered
errors Those answered with a 5xx status, or that ended with an error (an error that says its status counts by it, any other as a 500)
in flight The requests being answered now
latency The time from the filter's start to its end, in buckets

What that leaves out:

Percentiles

The latency of each request falls into a bucket. The upper bounds of the buckets start at 100 µs, and each is half as much again as the one before, up to about a minute; a last bucket takes what is slower.

A percentile is the upper bound of the bucket its request fell in. So p99 9.2 ms means: 99 % of the requests were answered within 9.2 ms, and the real figure is somewhere between that bound and the one before (two thirds of it). That is enough to tell 2 ms from 20 ms from 200 ms. It is not enough to compare 8 ms with 9 ms.

The total's percentiles are worked out from the buckets of all isolates added up, not averaged from the isolates' percentiles.

The window

RequestRates, the bean requestRates, lives in the application's first isolate. Every spring.web.debug.qps.log-interval it asks the isolates for their counts and keeps the difference to the last look: that is the window.

In code, the window is asked for like any answer of the first isolate, so the same code works in every isolate:

@RestController()
class LoadController {
  LoadController(this.primary);
  final PrimaryIsolate primary;

  // The last window as JSON: what /actuator/qps answers. null before the first.
  @GetMapping('/internal/load')
  Future<Object?> load() => primary.ask(RequestRates.handlerId);

  // Ends the window now: what /actuator/qps/now answers.
  @GetMapping('/internal/load/now')
  Future<Object?> now() => primary.ask(RequestRates.handlerId, 'now');
}

RequestRates.handlerId is web.qps. See Asking the first isolate.

A bean of the first isolate can take RequestRates itself and read the typed values:

@Component()
@ConditionalOnPrimaryIsolate()
class RateWatch {
  RateWatch(this.rates);
  final RequestRates rates;

  @Scheduled(fixedRate: Duration(minutes: 1))
  void check() {
    final p99 = rates.last?.total.percentile(0.99);
    if (p99 != null && p99 > const Duration(seconds: 1)) {
      alerts.slow(p99);
    }
  }
}
Member Meaning
RequestRates.last The last window, a RequestRateWindow; null before the first interval has passed
RequestRates.look() Ends the window now, logs it and answers it
RequestRateWindow.total All isolates together, a RequestRate
RequestRateWindow.isolates Each isolate by name, the first isolate first
RequestRate.qps, requests, errors, active, seconds The numbers of the window
RequestRate.percentile(share) The latency under which that share of the requests were answered (0.5, 0.99); null without requests

The RequestRates bean exists in the first isolate only. A bean that takes it must be there only too (@ConditionalOnPrimaryIsolate()), or the other isolates do not start; a bean of every isolate asks through PrimaryIsolate.

Out of the logs

The line is logged at info. To keep the counting and the endpoint without the lines:

logging:
  level:
    spring.web.qps: warn

The properties

Key Type Default Meaning
spring.web.debug.qps.enabled bool false Whether the application counts its requests. Read by the generator: without true the build has none of the feature's code
spring.web.debug.qps.log-interval Duration 10s The length of a window: how often the rate is worked out and logged. 0: nothing is logged, and a window is as long as the time between two looks (look(), /actuator/qps/now)

In Spring (Java)

Spring's counterpart is Micrometer's http.server.requests timer, read through the actuator's metrics endpoint or a monitoring system. This is smaller on purpose: one window, one line, and no code in the build unless you ask for it.

See also