Skip to content

Graduate-Level Google-Proof Q&A

A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Designed to be difficult even for skilled non-experts with access to Google.

117 models

Higher is better.

  1. 1GPT-6 AstraOpenAI96.0
  2. 2GPT-5.6 SolOpenAI94.6
  3. 3Claude Opus 4.7 (Adaptive)Anthropic94.2
  4. 4Claude Mythos 5Anthropic94.1
  5. 5Claude Opus 4.8Anthropic93.6
  6. 6GPT-5.5OpenAI93.6
  7. 7Kimi K3Moonshot AI93.5
  8. 8GPT-5.6 TerraOpenAI92.9
  9. 9GPT-5.4OpenAI92.8
  10. 10Ornith-1.5-397BOrnith AI92.8
  11. 11Qwen3.8 MaxAlibaba92.6
  12. 12Qwen3.7 MaxAlibaba92.4
  13. 13GPT-5.2OpenAI92.4
  14. 14GPT-5.6 LunaOpenAI92.3
  15. 15Hy4 previewTencent92.3
  16. 16Gemini 3.5 FlashGoogle92.2
  17. 17Qwen3.8-Flash-NextAlibaba91.7
  18. 18Claude Opus 4.6Anthropic91.3
  19. 19GLM-5.2Z.AI91.2
  20. 20Qwen3.8-Omni-FlashAlibaba91.0
  21. 21DeepSeek V4.1 Flash 1017DeepSeek90.9
  22. 22Kimi K2.6Moonshot AI90.5
  23. 23Qwen3.6 PlusAlibaba90.4
  24. 24Qwen3.7 PlusAlibaba90.3
  25. 25DeepSeek V4 Pro 0813DeepSeek90.1
  26. 26Grok 4.3xAI90.1
  27. 27Claude Sonnet 4.6Anthropic89.9
  28. 28Interfaze BetaInterfaze89.9
  29. 29Inkling-SmallThinking Machines Lab89.5
  30. 30Qwen3.8-27BAlibaba89.2
  31. 31Ornith-1.5-35B-A3BOrnith AI89.2
  32. 32Qwen3.5 397BAlibaba88.4
  33. 33DeepSeek V4 Flash 0731DeepSeek88.1
  34. 34GPT-5.4 miniOpenAI88.0
  35. 35InklingThinking Machines Lab87.9
  36. 36Qwen3.6-27BAlibaba87.8
  37. 37Kimi K2.5 (Reasoning)Moonshot AI87.6
  38. 38Kimi K2.5Moonshot AI87.6
  39. 39Hy3 PreviewTencent87.2
  40. 40Claude Opus 4.5Anthropic87.0
Higher is better. Showing the top 40 of 117. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Multiple choice questions
Tasks
448 questions
Authors
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, Samuel R. Bowman (2023)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.

All results

1GPT-6 Astra OpenAI96.0
2GPT-5.6 Sol OpenAI94.6
3Claude Opus 4.7 (Adaptive) Anthropic94.2
4Claude Mythos 5 Anthropic94.1
5Claude Opus 4.8 Anthropic93.6
6GPT-5.5 OpenAI93.6
7Kimi K3 Moonshot AI93.5
8GPT-5.6 Terra OpenAI92.9
9GPT-5.4 OpenAI92.8
10Ornith-1.5-397B Ornith AI92.8
11Qwen3.8 Max Alibaba92.6
12Qwen3.7 Max Alibaba92.4
13GPT-5.2 OpenAI92.4
14GPT-5.6 Luna OpenAI92.3
15Hy4 preview Tencent92.3
16Gemini 3.5 Flash Google92.2
17Qwen3.8-Flash-Next Alibaba91.7
18Claude Opus 4.6 Anthropic91.3
19GLM-5.2 Z.AI91.2
20Qwen3.8-Omni-Flash Alibaba91.0
21DeepSeek V4.1 Flash 1017 DeepSeek90.9
22Kimi K2.6 Moonshot AI90.5
23Qwen3.6 Plus Alibaba90.4
24Qwen3.7 Plus Alibaba90.3
25DeepSeek V4 Pro 0813 DeepSeek90.1
26Grok 4.3 xAI90.1
27Claude Sonnet 4.6 Anthropic89.9
28Interfaze Beta Interfaze89.9
29Inkling-Small Thinking Machines Lab89.5
30Qwen3.8-27B Alibaba89.2
31Ornith-1.5-35B-A3B Ornith AI89.2
32Qwen3.5 397B Alibaba88.4
33DeepSeek V4 Flash 0731 DeepSeek88.1
34GPT-5.4 mini OpenAI88.0
35Inkling Thinking Machines Lab87.9
36Qwen3.6-27B Alibaba87.8
37Kimi K2.5 (Reasoning) Moonshot AI87.6
38Kimi K2.5 Moonshot AI87.6
39Hy3 Preview Tencent87.2
40Claude Opus 4.5 Anthropic87.0
41Nemotron 3 Ultra NVIDIA87.0
42Qwen3.5-122B-A10B Alibaba86.6
43Ornith-1.5-9B Ornith AI86.4
44GLM-5 Z.AI86.0
45Qwen3.6-35B-A3B Alibaba86.0
46Ternary Bonsai 2 27B Prism ML85.8
47GLM-4.7 Z.AI85.7
48Qwen3.5-27B Alibaba85.5
49Ling 3.0 Flash InclusionAI85.0
50Gemma 4 31B Google84.3
51MAI-Thinking-1 Microsoft84.2
52Qwen3.5-35B-A3B Alibaba84.2
53MiMo-V2-Flash Xiaomi83.7
54Claude Sonnet 4.5 Anthropic83.4
55Gemini 2.5 Pro Google83.0
56GPT-5.4 nano OpenAI82.8
57Gemma 4 26B A4B Google82.3
58Grok 3 Mini xAI80.3
59DeepSeek V3.1 (Reasoning) DeepSeek80.1
60GPT-OSS 120B OpenAI80.1
61Nemotron 3 Super 100B NVIDIA79.2
62o1-pro OpenAI79.0
63Gemma 4 12B Google78.8
64Qwen3 235B 2507 Alibaba77.5
65o3-mini OpenAI77.2
66Nemotron Ultra 253B NVIDIA76.0
67o1 OpenAI75.7
68Nemotron 3.5 Lightning 30B A3B NVFP4 NVIDIA75.6
69Grok 3 [Beta] xAI75.4
70GLM-4.7-Flash Z.AI75.2
71Kimi K2 Moonshot AI75.1
72DeepSeek V3.1 DeepSeek74.9
73Claude 4 Opus Anthropic74.9
74LFM2.5-24B-A2B Liquid AI74.4
75Nemotron 3 Nano 30B NVIDIA73.0
76Nemotron 3 Nano Omni 30B A3B NVIDIA72.2
77DeepSeek-R1 DeepSeek71.5
78GPT-OSS 20B OpenAI71.5
79ZAYA1-8B Zyphra71.0
80Magistral Medium Mistral70.8
81GPT-4.5 OpenAI69.5
82Claude 3.7 Sonnet Anthropic68.0
83GPT-4.1 OpenAI66.3
84Phi-4-reasoning Microsoft65.8
85QwQ-32B Alibaba65.6
86Gemini 2.0 Flash Google65.2
87GPT-4.1 mini OpenAI64.2
88Granite 4.2 8B IBM64.1
89Qwen3 235B-A22B Alibaba62.9
90Qwen2.5-Max Alibaba60.1
91o1-mini OpenAI60.0
92Claude 3.5 Sonnet Anthropic59.4
93DeepSeek V3 DeepSeek59.1
94Ling 2.6 Flash InclusionAI59.0
95Gemma 4 E4B Google58.6
96ZAYA1-74B-Preview Zyphra57.3
97Phi-4 Microsoft56.1
98Grok 2 xAI56.0
99Qwen3 32B Alibaba54.6
100Llama 3.3 70B Meta50.5
101GPT-4.1 nano OpenAI50.3
102Gemini 1.5 Flash Google50.0
103Llama 3.1 70B Meta46.7
104Qwen2.5-VL-32B Alibaba46.0
105Mistral Small 3.1 Mistral46.0
106Gemma 4 E2B Google43.4
107Soofi S 30B-A3B Soofi Project43.4
108Claude 3.5 Haiku Anthropic41.6
109Mellum2-12B-A2.5B-Instruct JetBrains40.9
110LFM2.5-350M Liquid AI30.6
111Phi-3.5-mini Microsoft30.4
112Llama 3.1 8B Meta30.4
113Granite-4.0-H-1B IBM29.7
114Granite-4.0-H-350M IBM26.1
115LFM2.5-VL-450M Liquid AI25.7
116LFM2.5-230M Liquid AI25.4
117Granite-4.0-350M IBM24.1