Skip to content

Instruction-Following Eval

A benchmark of 541 prompts built from 25 verifiable instruction types. It tests whether a model follows checkable constraints such as keyword, length, casing, and response-format requirements.

53 models

Higher is better.

  1. 1LFM2.5-24B-A2BLiquid AI96.8
  2. 2Qwen3.5-27BAlibaba95.0
  3. 3Agents-A1InternScience94.8
  4. 4Qwen3.7 PlusAlibaba94.6
  5. 5Qwen3.7 MaxAlibaba94.3
  6. 6Qwen3.6 PlusAlibaba94.3
  7. 7dots3-note PreviewDots Studio93.9
  8. 8Kimi K2.5Moonshot AI93.9
  9. 9o3-miniOpenAI93.9
  10. 10Qwen3.5-122B-A10BAlibaba93.4
  11. 11Qwen3.5 397BAlibaba92.6
  12. 12GLM-5Z.AI92.6
  13. 13K-EXAONE 2.0LG AI Research92.4
  14. 14o1OpenAI92.2
  15. 15Llama 3.3 70BMeta92.1
  16. 16Qwen3.5-35B-A3BAlibaba91.9
  17. 17LFM2.5-8B-A1BLiquid AI91.8
  18. 18Ternary Bonsai 2 27BPrism ML91.3
  19. 19Claude Opus 4.5Anthropic90.9
  20. 20Claude 3.7 SonnetAnthropic90.8
  21. 21Mellum2.1-12B-A2.5B-ThinkingJetBrains90.6
  22. 22Kimi K2Moonshot AI89.8
  23. 23Nemotron Ultra 253BNVIDIA89.5
  24. 24Qwen3 235B 2507Alibaba88.7
  25. 25Llama 3.1 405BMeta88.6
  26. 26GPT-4.1 miniOpenAI88.5
  27. 27Llama 3.1 70BMeta87.5
  28. 28GPT-4.1OpenAI87.4
  29. 29MiniCPM5-2BOpenBMB86.7
  30. 30DeepSeek V3DeepSeek86.1
  31. 31Claude 3.5 HaikuAnthropic85.9
  32. 32ZAYA1-8BZyphra85.6
  33. 33Sarvam 105BSarvam84.8
  34. 34QwQ-32BAlibaba83.9
  35. 35Phi-4-reasoningMicrosoft83.4
  36. 36DeepSeek-R1DeepSeek83.3
  37. 37GPT-4.1 nanoOpenAI83.2
  38. 38Qwen3 235B-A22BAlibaba83.2
  39. 39Qwen3 32BAlibaba83.2
  40. 40LFM2.5-VL-3BLiquid AI82.3
Higher is better. Showing the top 40 of 53. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Constrained generation
Tasks
541 prompts across 25 instruction types
Authors
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, Le Hou (2023)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.

All results

1LFM2.5-24B-A2B Liquid AI96.8
2Qwen3.5-27B Alibaba95.0
3Agents-A1 InternScience94.8
4Qwen3.7 Plus Alibaba94.6
5Qwen3.7 Max Alibaba94.3
6Qwen3.6 Plus Alibaba94.3
7dots3-note Preview Dots Studio93.9
8Kimi K2.5 Moonshot AI93.9
9o3-mini OpenAI93.9
10Qwen3.5-122B-A10B Alibaba93.4
11Qwen3.5 397B Alibaba92.6
12GLM-5 Z.AI92.6
13K-EXAONE 2.0 LG AI Research92.4
14o1 OpenAI92.2
15Llama 3.3 70B Meta92.1
16Qwen3.5-35B-A3B Alibaba91.9
17LFM2.5-8B-A1B Liquid AI91.8
18Ternary Bonsai 2 27B Prism ML91.3
19Claude Opus 4.5 Anthropic90.9
20Claude 3.7 Sonnet Anthropic90.8
21Mellum2.1-12B-A2.5B-Thinking JetBrains90.6
22Kimi K2 Moonshot AI89.8
23Nemotron Ultra 253B NVIDIA89.5
24Qwen3 235B 2507 Alibaba88.7
25Llama 3.1 405B Meta88.6
26GPT-4.1 mini OpenAI88.5
27Llama 3.1 70B Meta87.5
28GPT-4.1 OpenAI87.4
29MiniCPM5-2B OpenBMB86.7
30DeepSeek V3 DeepSeek86.1
31Claude 3.5 Haiku Anthropic85.9
32ZAYA1-8B Zyphra85.6
33Sarvam 105B Sarvam84.8
34QwQ-32B Alibaba83.9
35Phi-4-reasoning Microsoft83.4
36DeepSeek-R1 DeepSeek83.3
37GPT-4.1 nano OpenAI83.2
38Qwen3 235B-A22B Alibaba83.2
39Qwen3 32B Alibaba83.2
40LFM2.5-VL-3B Liquid AI82.3
41Ternary Bonsai 8B Prism ML81.8
42Kanana-2 3B Instruct Kakao81.0
43Celeris-1 Celeris80.8
44MiniCPM5-1B OpenBMB80.4
45Llama 3.1 8B Meta80.4
461-bit Bonsai 8B Prism ML79.8
47Granite-4.0-H-1B IBM78.5
48LFM2.5-350M Liquid AI77.0
49Mellum2-12B-A2.5B-Instruct JetBrains75.8
50LFM2.5-230M Liquid AI71.7
51Granite-4.0-H-350M IBM61.6
52LFM2.5-VL-450M Liquid AI61.2
53Granite-4.0-350M IBM55.4