FrontierSWE

An ultra-long-horizon software-engineering benchmark with open-ended implementation, performance, and research tasks designed to challenge frontier coding agents.

3 models

Higher is better.

1Kimi K3Moonshot AI81.2
2GLM-5.3Z.AI78.1
3Qwen3.8 MaxAlibaba73.5

Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric: Mean@5, best@5, average rank, and dominance
Tasks: 17 ultra-long-horizon engineering and research tasks
Authors: Evan Chu, Rajan Agarwal, Abishek Thangamuthu, Brendan Graham, Justus Mattern (2026)
Source: FrontierSWE: Benchmarking coding agents at the limits of human abilities ↗

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.