Skip to content

CWE-bench v1

Defensive vulnerability patching on 120 held-out tasks spanning 73 weakness types.

11 models

Higher is better.

  1. 1GPT-6 AstraOpenAI68.0
  2. 2Gemini 4 ArgonGoogle68.0
  3. 3Grok 4.7xAI68.0
  4. 4Claude Opus 5.5Anthropic67.0
  5. 5Claude Fable 5.1Anthropic58.0
  6. 6Grok 4.6xAI57.0
  7. 7DeepSeek V4.1 Flash 1017DeepSeek55.0
  8. 8Muse Spark 1.3Meta55.0
  9. 9Hy4 previewTencent53.0
  10. 10GPT-6 SolOpenAI52.0
  11. 11InklingThinking Machines Lab37.0
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Pass@1
Tasks
Audit and patch held-out C/C++, Go, Java, TypeScript/JavaScript, Python, and Rust repositories
Authors
Collinear AI (2026)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.