Skip to content

Bug Hunt Bench

A blind-graded coding-agent benchmark with 105 planted bugs across two production TypeScript repositories.

23 models

Higher is better.

  1. 1Claude Sonnet 5.5Anthropic51.3
  2. 2GPT-6 AstraOpenAI45.0
  3. 3GPT-6.1 SolOpenAI44.3
  4. 4Claude Fable 5.1Anthropic43.0
  5. 5GPT-5.6 SolOpenAI42.0
  6. 6Muse Wonder FullMeta42.0
  7. 7Claude Opus 5.5Anthropic41.7
  8. 8Gemini 4 ArgonGoogle40.7
  9. 9DeepSeek V4.1 Flash 1017DeepSeek39.1
  10. 10Muse Spark 1.3Meta32.2
  11. 11Muse WonderMeta31.7
  12. 12Grok 4.7xAI29.0
  13. 13Claude Opus 5Anthropic27.0
  14. 14Grok 4.6xAI27.0
  15. 15Qwen3.8 MaxAlibaba25.7
  16. 16MiMo-V2.6-ProXiaomi22.7
  17. 17Claude Haiku 5.5Anthropic21.5
  18. 18Kimi K3Moonshot AI21.0
  19. 19GLM-5.3Z.AI19.0
  20. 20GPT-6 LunaOpenAI18.3
  21. 21Gemini 3.8 FlashGoogle18.0
  22. 22GLM-5.3-FlashZ.AI17.7
  23. 23Qwen3.8-27BAlibaba15.0
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Strict planted bugs fixed
Tasks
105 planted bugs across two production TypeScript repositories
Authors
Pawel Huryn (2026)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.