Skip to content

Hermes Bench: 150 agent tasks graded on the files and state the agent leaves behind

Tests agents on 150 tasks across 25 categories, including Hermes skills, research, diagrams and art, memory, tool use and safety. Each agent starts in a workspace with real files, some tasks add follow-up turns, and the grader checks the files and state left behind.

16 models

Higher is better.

  1. 1Claude Opus 5.5Anthropic76.7
  2. 2Grok 4.7xAI74.4
  3. 3GPT-6.1 SolOpenAI73.9
  4. 4GPT-6 AstraOpenAI73.7
  5. 5GPT-6 SolOpenAI73.5
  6. 6Claude Sonnet 5.5Anthropic73.5
  7. 7GLM-5.3Z.AI71.8
  8. 8Kimi K3Moonshot AI71.5
  9. 9Claude Haiku 5.5Anthropic71.0
  10. 10Qwen3.8 MaxAlibaba70.5
  11. 11GPT-6 LunaOpenAI70.5
  12. 12Step 5 PreviewStepFun69.2
  13. 13GLM-5.3-FlashZ.AI68.9
  14. 14Gemini 3.8 FlashGoogle62.2
  15. 15Hy4 previewTencent58.6
  16. 16Ling 3.0 FlashInclusionAI53.8
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
pass@1 score
Tasks
150 tasks across 25 categories
Authors
Nous Research (2026)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.