How we measure model speed

Speed is the one dimension ModelBench measures itself rather than compiles. Here is exactly how.

Status, October 2026: the first measurements have not been published yet. This is the method they will use; until then, speed columns show a dash.

The test

For each model we stream one fixed prompt, asking for a ~350-word explanation, at temperature 0 with up to 2,000 output tokens, through OpenRouter. We repeat it several times and report the median.

What we record

• Time to first token (TTFT): the first streamed token of any kind, including a reasoning model's thinking.
• Time to first answer token: the first visible answer token. For reasoning models this includes all the thinking time, which is what a user actually waits for.
• Output speed: output tokens ÷ (total time − time to first token), in tokens per second.
• 500-token response time: time to first answer token + 500 tokens at the measured speed. It is a single number for "how long until I have a full answer?"

Caveats

• OpenRouter routes requests to one of several hosting providers, and speed varies by provider and by load. Model pages list the providers seen.
• A handful of trials gives a reasonable median, not a precise one; numbers move between runs.
• Measurements cost real money, so we measure the most popular and highest-ranked models first and expand from there.

When a model has no measurement, we show a dash. We never fill gaps with figures from other sites.