Skip to content

SWE-Atlas Codebase QnA

A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.

4 models

Higher is better.

  1. 1Ling 3.1 FlashInclusionAI64.5
  2. 2Muse Spark 1.3Meta59.4
  3. 3Muse Spark OpenMeta58.0
  4. 4Beam 1.0 501B-A23BReflection AI31.8
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Mean pass@1
Tasks
124 codebase questions across 11 repositories
Authors
Meta Superintelligence Labs and Scale AI (2026)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.