ExtractBench Pro: long-tail document extraction challenges (LlamaIndex)

A harder companion to ExtractBench: 100 synthetic documents and 2,622 pages across 86 document types that test document conventions, house vocabulary, counting rules, drawn relations and charts, scored by perfect-document rate.

5 models

Higher is better.

1GPT-6 AstraOpenAI58
2Claude Opus 5.5Anthropic51
3Claude Sonnet 5.5Anthropic32
4GPT-6.1 SolOpenAI12
5GPT-6 LunaOpenAI0

Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric: Perfect-document rate, value F1, word and page grounding F1 and cost, overall and per challenge
Tasks: 100 synthetic documents, 2,622 pages, 86 document types, six challenge groups
Authors: LlamaIndex (2026)
Source: ExtractBench Pro leaderboard and challenge descriptions ↗

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.