SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration?

Tests whether coding agents can complete long-horizon, whole-repository stack migrations while preserving the original program's behavior.

1 models

Higher is better.

Higher is better. Bar colour: blue proprietary, orange open weights, green other.

Metric: Migration audit, frozen behavioral checks, and agentic verification
Tasks: 20 whole-repository stack migrations
Authors: Deyao Hong, Yizhe Chi, Wenyi Li, Xiaoqiu Wang, Mingju Gao, Kaisen Yang, Bingxiang He, Youjie Zheng, Calvin Xiao, Qinhuai Na (2026)
Source: SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration? ↗

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.