Skip to content

CharXiv Reasoning

A scientific chart reasoning benchmark that tests whether models can understand, interpret, and reason about complex scientific visualizations including plots, diagrams, and data charts.

42 models

Higher is better.

  1. 1Qwen3.8 MaxAlibaba93.5
  2. 2Claude Mythos 5Anthropic93.5
  3. 3Qwen3.8-Omni-FlashAlibaba91.4
  4. 4Kimi K3Moonshot AI91.3
  5. 5Claude Opus 4.7 (Adaptive)Anthropic91.0
  6. 6Qwen3.8-Flash-NextAlibaba90.6
  7. 7Qwen3.8-27BAlibaba90.2
  8. 8Claude Opus 4.8Anthropic89.9
  9. 9GLM-5.3-FlashZ.AI89.4
  10. 10Gemini 3.7 FlashGoogle88.7
  11. 11Muse Spark 1.1Meta88.4
  12. 12Claude Sonnet 5Anthropic88.3
  13. 13Muse SparkMeta86.4
  14. 14Qwen3.7 PlusAlibaba85.9
  15. 15Seed 2.1 ProByteDance85.4
  16. 16Gemini 3.5 FlashGoogle84.2
  17. 17GPT-5.4OpenAI82.8
  18. 18MiniMax M3.1 FlashMiniMax82.5
  19. 19GPT-5.2OpenAI82.1
  20. 20InklingThinking Machines Lab82.0
  21. 21Qwen3.6 PlusAlibaba81.5
  22. 22Gemini 3 ProGoogle81.4
  23. 23Inkling-SmallThinking Machines Lab81.3
  24. 24MiMo-V2.5Xiaomi81.0
  25. 25Nemotron 3.5 Super VL 120B A12BNVIDIA80.8
  26. 26Qwen3.5 397BAlibaba80.8
  27. 27Kimi K2.6Moonshot AI80.4
  28. 28Gemini 3.1 ProGoogle80.2
  29. 29Qwen3.5-27BAlibaba79.5
  30. 30Muse Glimmer 30BMeta78.8
  31. 31Qwen3.6-27BAlibaba78.4
  32. 32Qwen3.6-35B-A3BAlibaba78.0
  33. 33Kimi K2.5 (Reasoning)Moonshot AI77.5
  34. 34Kimi K2.5Moonshot AI77.5
  35. 35Qwen3.5-35B-A3BAlibaba77.5
  36. 36Claude Sonnet 4.6Anthropic77.4
  37. 37Qwen3.5-122B-A10BAlibaba77.2
  38. 38Nemotron 3 Nano Omni 30B A3BNVIDIA76.3
  39. 39Gemini 3.1 Flash-LiteGoogle73.2
  40. 40Claude Opus 4.5Anthropic68.5
Higher is better. Showing the top 40 of 42. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Chart understanding and reasoning
Tasks
Scientific chart reasoning
Authors
CharXiv authors (2024)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.

All results

1Qwen3.8 Max Alibaba93.5
2Claude Mythos 5 Anthropic93.5
3Qwen3.8-Omni-Flash Alibaba91.4
4Kimi K3 Moonshot AI91.3
5Claude Opus 4.7 (Adaptive) Anthropic91.0
6Qwen3.8-Flash-Next Alibaba90.6
7Qwen3.8-27B Alibaba90.2
8Claude Opus 4.8 Anthropic89.9
9GLM-5.3-Flash Z.AI89.4
10Gemini 3.7 Flash Google88.7
11Muse Spark 1.1 Meta88.4
12Claude Sonnet 5 Anthropic88.3
13Muse Spark Meta86.4
14Qwen3.7 Plus Alibaba85.9
15Seed 2.1 Pro ByteDance85.4
16Gemini 3.5 Flash Google84.2
17GPT-5.4 OpenAI82.8
18MiniMax M3.1 Flash MiniMax82.5
19GPT-5.2 OpenAI82.1
20Inkling Thinking Machines Lab82.0
21Qwen3.6 Plus Alibaba81.5
22Gemini 3 Pro Google81.4
23Inkling-Small Thinking Machines Lab81.3
24MiMo-V2.5 Xiaomi81.0
25Nemotron 3.5 Super VL 120B A12B NVIDIA80.8
26Qwen3.5 397B Alibaba80.8
27Kimi K2.6 Moonshot AI80.4
28Gemini 3.1 Pro Google80.2
29Qwen3.5-27B Alibaba79.5
30Muse Glimmer 30B Meta78.8
31Qwen3.6-27B Alibaba78.4
32Qwen3.6-35B-A3B Alibaba78.0
33Kimi K2.5 (Reasoning) Moonshot AI77.5
34Kimi K2.5 Moonshot AI77.5
35Qwen3.5-35B-A3B Alibaba77.5
36Claude Sonnet 4.6 Anthropic77.4
37Qwen3.5-122B-A10B Alibaba77.2
38Nemotron 3 Nano Omni 30B A3B NVIDIA76.3
39Gemini 3.1 Flash-Lite Google73.2
40Claude Opus 4.5 Anthropic68.5
41Grok 4.20 xAI60.9
42Command A+ Cohere52.7