Skip to content

Claude 3.5 Haiku

AnthropicProprietaryNon-reasoningReleased Oct 22, 2024
Compare with Claude Opus 5.5Source scores ↗
Overall score
—
Not enough evidence to rank yet
Context window
200K
200,000 tokens
Price per 1M tokens
— in · — out
No price listed
Blended price
—
3:1 input:output tokens
Output speed
—
Not measured yet

Category scores

Scores per capability area with the model's rank among ranked models in that category.

  1. 1Instruction following#9450.1
  2. 2Knowledge49.2
  3. 3Reasoning43.8
  4. 4Multimodal31.7
  5. 5Coding24.5
Higher is better. #N is the model's rank in that category, where it is eligible to be ranked.

Benchmark results (10)

As reported in system cards, papers and public evaluations. Position compares this model with every other model reporting the same benchmark.

Knowledge

GPQA41.6#108 of 117
MMLU-Pro65.0#56 of 67
MMLU80.3#20 of 28

Coding

SWE-bench Verified40.6#94 of 98
HumanEval88.1#9 of 18

Reasoning

BBH86.6#1 of 7
DROP83.1#3 of 5

Multilingual

MGSM85.6#5 of 11

Instruction following

IFEval85.9#31 of 53

Math

AIME 20245.3#19 of 19

Pricing by source

USD per 1M tokens.

SourceInputOutputCached in
List price———

Arena ratings

Human-preference ratings from LMArena. Best-rated variant shown.

Vision(20241022)1093#132 · 1.4K votes

Speed

Measured by us through OpenRouter (median of trials).

Not available on OpenRouter, so we cannot measure it yet.

Related models

Same family