
# Requests per second

The debug feature `qps` counts the requests the server answers and works out, every few seconds, the requests per second, how long they took, how many are in flight and how many failed: for each isolate and for all together. It logs one line per interval, and the actuator's [`qps` endpoint](../../spring-boot-actuator/qps.md) answers the same numbers as JSON, from whichever isolate takes the request.

It is in the build only with `spring.web.debug.qps.enabled: true`. Without that line the application has no code that counts its requests.

## Why

"How much does the server take?" is the first question of a load test and of an evening when the server is slow. This answers it in one line, while it happens, and per isolate, so a [pool](isolates.md) that serves unevenly shows.

It is a debug feature, not a metrics system. It keeps one window, the last, and nothing over time.

## Example

```yaml
# lib/resources/application.yml
spring:
  web:
    debug:
      qps:
        enabled: true       # read by the build
        log-interval: 10s
```

```bash
dart run build_runner build   # the property is read here
dart run bin/main.dart
```

Under load, the first isolate logs on `spring.web.qps`, at `info`:

```text
1250.4 requests/s (p50 1.1 ms, p99 9.2 ms), 14 in flight, 0 errors; main 412.0, isolate-1 420.2, isolate-2 418.2
```

With one isolate the line ends after the errors. In an interval without a request, and with none in flight, nothing is logged.

## The build decides

`spring.web.debug.qps.enabled` is a [build property](../../spring-core-dart-generator/optimizations/build-conditions.md): the generator reads it when `build_runner` runs and puts the feature's module into the application, or does not.

| `spring.web.debug.qps.enabled` | The application |
|---|---|
| `true` | Has the filter that counts and `RequestRates`, in every build |
| `false`, or not set | Has none of that code |

- Set it in `application.yml` or `application.properties`, without a profile: the build does not read profile files, environment variables or `--dart-define` values.
- Run `build_runner` again after changing it.
- **A property at run time cannot switch it on or off.** The module has no runtime condition: what the build put in, runs.
- The module is native only, like the web starter itself. A browser build has none of it.

## What is counted

`RequestRateFilter`, the bean `requestRateFilter`, is the first filter of the chain, around everything else: the security filter chain, the handler, the writing of the answer. For each request it counts:

| Count | Meaning |
|---|---|
| requests | The requests answered |
| errors | Those answered with a `5xx` status, or that ended with an error (an error that says its status counts by it, any other as a `500`) |
| in flight | The requests being answered now |
| latency | The time from the filter's start to its end, in buckets |

What that leaves out:

- **The time a request waited for the isolate.** The filter starts when the isolate gets to the request. A request that waited 200 ms behind others and was answered in 2 ms counts as 2 ms. The wait shows as [lag](../../spring-core-dart/isolates/load.md#how-lag-and-busy-are-measured), not here.
- **Requests refused because the server was full.** Those are answered before any filter ([Overload](overload.md)). They are the `refused` of the task manager.
- **The network.** The time is the server's, not the client's.

## Percentiles

The latency of each request falls into a bucket. The upper bounds of the buckets start at 100 µs, and each is half as much again as the one before, up to about a minute; a last bucket takes what is slower.

A percentile is the upper bound of the bucket its request fell in. So `p99 9.2 ms` means: 99 % of the requests were answered within 9.2 ms, and the real figure is somewhere between that bound and the one before (two thirds of it). That is enough to tell 2 ms from 20 ms from 200 ms. It is not enough to compare 8 ms with 9 ms.

The total's percentiles are worked out from the buckets of all isolates added up, not averaged from the isolates' percentiles.

## The window

`RequestRates`, the bean `requestRates`, lives in the application's first isolate. Every `spring.web.debug.qps.log-interval` it asks the isolates for their counts and keeps the difference to the last look: that is the window.

- Requests per second are the requests of the window over its length.
- A request counts in the window in which it was answered.
- An isolate that was started again counts from zero; its window is what it has now.
- An isolate added or removed by [scaling](scaling.md) appears in or leaves the line.
- `in flight` is the number at the end of the window.

In code, the window is asked for like any answer of the first isolate, so the same code works in every isolate:

```dart
@RestController()
class LoadController {
  LoadController(this.primary);
  final PrimaryIsolate primary;

  // The last window as JSON: what /actuator/qps answers. null before the first.
  @GetMapping('/internal/load')
  Future<Object?> load() => primary.ask(RequestRates.handlerId);

  // Ends the window now: what /actuator/qps/now answers.
  @GetMapping('/internal/load/now')
  Future<Object?> now() => primary.ask(RequestRates.handlerId, 'now');
}
```

`RequestRates.handlerId` is `web.qps`. See [Asking the first isolate](../../spring-core-dart/isolates/primary-isolate.md).

A bean of the first isolate can take `RequestRates` itself and read the typed values:

```dart
@Component()
@ConditionalOnPrimaryIsolate()
class RateWatch {
  RateWatch(this.rates);
  final RequestRates rates;

  @Scheduled(fixedRate: Duration(minutes: 1))
  void check() {
    final p99 = rates.last?.total.percentile(0.99);
    if (p99 != null && p99 > const Duration(seconds: 1)) {
      alerts.slow(p99);
    }
  }
}
```

| Member | Meaning |
|---|---|
| `RequestRates.last` | The last window, a `RequestRateWindow`; `null` before the first interval has passed |
| `RequestRates.look()` | Ends the window now, logs it and answers it |
| `RequestRateWindow.total` | All isolates together, a `RequestRate` |
| `RequestRateWindow.isolates` | Each isolate by name, the first isolate first |
| `RequestRate.qps`, `requests`, `errors`, `active`, `seconds` | The numbers of the window |
| `RequestRate.percentile(share)` | The latency under which that share of the requests were answered (`0.5`, `0.99`); `null` without requests |

The `RequestRates` bean exists in the first isolate only. A bean that takes it must be there only too (`@ConditionalOnPrimaryIsolate()`), or the other isolates do not start; a bean of every isolate asks through `PrimaryIsolate`.

## Out of the logs

The line is logged at `info`. To keep the counting and the endpoint without the lines:

```yaml
logging:
  level:
    spring.web.qps: warn
```

## The properties

| Key | Type | Default | Meaning |
|---|---|---|---|
| `spring.web.debug.qps.enabled` | `bool` | `false` | Whether the application counts its requests. Read by the generator: without `true` the build has none of the feature's code |
| `spring.web.debug.qps.log-interval` | `Duration` | `10s` | The length of a window: how often the rate is worked out and logged. `0`: nothing is logged, and a window is as long as the time between two looks (`look()`, `/actuator/qps/now`) |

## In Spring (Java)

Spring's counterpart is Micrometer's `http.server.requests` timer, read through the actuator's `metrics` endpoint or a monitoring system. This is smaller on purpose: one window, one line, and no code in the build unless you ask for it.

## See also

- [The `qps` endpoint](../../spring-boot-actuator/qps.md)
- [A pool of isolates](isolates.md)
- [Overload](overload.md)
- [Filters](../filters.md)
- [The load of the isolates](../../spring-core-dart/isolates/load.md)
- [Asking the first isolate](../../spring-core-dart/isolates/primary-isolate.md)
- [Modules left out of the build](../../spring-core-dart-generator/optimizations/build-conditions.md)
- [Load testing](../../../docs/load-testing.md)
