Performance measurements
Request history shows total duration. Provider tables show time to first token for text and total duration for Transcribe. These measure different parts of a request.
| Parameter | Type | Default | Description |
|---|---|---|---|
| latency_ms | integer | null | — | Total duration, including queueing, uploads, processing and retries. Kept for API compatibility. |
| ttft_ms | integer | null | — | Time until the model starts returning reasoning, an answer or tool-call output. |
| generation_ms | integer | null | — | First generated output to last generated output. |
GET /v1/usage and request details return these fields. GET /v1/usage/summary adds averages, p50, p95 and sample counts for each measurement. Historical requests, cache hits and failed requests have no provider timing. JSON-only BYOK completions have no TTFT measurement.
Generation speed
Generation speed uses the sum of output tokens minus one per measured request, divided by summed first-to-last output time. It excludes incomplete responses and requests with fewer than two output tokens or zero generation time. Check generation_samples before interpreting tps; zero samples means no measurement.
Model performance
GET /api/models/catalog/{lab}/{slug}/performance returns the figures on each model page, per provider. It needs no key, is refreshed every 5 minutes and cached for a minute.
| Parameter | Type | Default | Description |
|---|---|---|---|
| window | string | 7d | 1d, 7d or 30d. Points are hourly, every 6 hours or daily. |
| percentile | integer | 50 | 50, 75, 90, 95 or 99. For throughput it is the speed that share of requests reached or beat. |
| effort | string | all | all, or one reasoning level. efforts in the response lists the levels with traffic. |
Each provider has a summary and a series with throughput (tokens per second), ttft and latency (milliseconds), toolCallErrorRate, structuredErrorRate and cacheHitRate. Rates run from 0 to 1, and a value is null when there were no requests to measure. A tool-call error is a call whose arguments are not a JSON object; a structured-output error is a JSON-mode reply that does not parse before repair.