Skip to content

Massive Multitask Language Understanding

A comprehensive multiple-choice question answering test covering 57 tasks including elementary mathematics, US history, computer science, law, and more. Tests knowledge across diverse academic subjects from high school to professional level.

28 models

Higher is better.

  1. 1o1OpenAI91.8
  2. 2DeepSeek-R1DeepSeek90.8
  3. 3GPT-4.5OpenAI90.8
  4. 4Sarvam 105BSarvam90.6
  5. 5GPT-4.1OpenAI90.2
  6. 6GPT-OSS 120BOpenAI90.0
  7. 7Kimi K2Moonshot AI89.5
  8. 8GPT-4.1 miniOpenAI87.5
  9. 9Grok 2xAI87.5
  10. 10o3-miniOpenAI86.9
  11. 11Llama 3.1 70BMeta86.0
  12. 12Llama 3.3 70BMeta86.0
  13. 13LongCat-Flash-Lite-SparseMeituan85.3
  14. 14GPT-OSS 20BOpenAI85.3
  15. 15o1-miniOpenAI85.2
  16. 16Sarvam 30BSarvam85.1
  17. 17Phi-4Microsoft84.8
  18. 18GPT-4o miniOpenAI82.0
  19. 19Mistral Small 3.1Mistral80.6
  20. 20Claude 3.5 HaikuAnthropic80.3
  21. 21GPT-4.1 nanoOpenAI80.1
  22. 22Mixtral 8x22B Instruct v0.1Mistral77.8
  23. 23Gemma 2 27BGoogle76.2
  24. 24Llama 3.1 8BMeta73.0
  25. 25Phi-3.5-miniMicrosoft69.0
  26. 26Granite-4.0-H-1BIBM59.7
  27. 27Granite-4.0-H-350MIBM36.2
  28. 28Granite-4.0-350MIBM35.0
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Multiple choice questions
Tasks
57 subjects
Authors
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, Jacob Steinhardt (2020)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.