Skip to content

ProgramBench: Can Language Models Rebuild Programs From Scratch?

A cleanroom software-engineering benchmark where agents receive only a compiled executable and documentation, then must architect and implement a complete codebase that reproduces the original program's behavior.

15 models

Higher is better.

  1. 1Claude Opus 5Anthropic93.0
  2. 2Claude Opus 5.5Anthropic91.2
  3. 3Claude Fable 5.1Anthropic87.6
  4. 4Step 5 PreviewStepFun80.5
  5. 5Claude Sonnet 5.5Anthropic79.7
  6. 6Kimi K3Moonshot AI77.8
  7. 7GLM-5.2Z.AI63.7
  8. 8Muse Wonder FullMeta59.0
  9. 9Kimi K2.7 CodeMoonshot AI53.6
  10. 10Muse WonderMeta50.0
  11. 11DeepSeek V4.1 Flash 1017DeepSeek29.1
  12. 12MiMo-V2.6-ProXiaomi26.5
  13. 13MiMo-V2.6-FlashXiaomi26.0
  14. 14GLM-5.3Z.AI19.0
  15. 15Hy4 previewTencent17.5
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Cleanroom executable reimplementation
Tasks
200 program reconstruction tasks
Authors
John Yang, Kilian Lieret, Jeffrey Ma, Parth Thakkar, Dmitrii Pedchenko, Sten Sootla, Emily McMilin, Pengcheng Yin, Rui Hou, Gabriel Synnaeve, Diyi Yang, Ofir Press (2026)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.