Skip to content

VulcanBench Coding Intelligence Index v1

A post-cutoff software-engineering benchmark with hidden functional tests and regression guards, reported for vendor coding-agent harnesses.

3 models

Higher is better.

  1. 1Claude Opus 5Anthropic96.4
  2. 2Claude Sonnet 5Anthropic89.2
  3. 3GPT-5.6 SolOpenAI86.5
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Pass@1 with vendor coding-agent harnesses
Tasks
38 validated post-cutoff repository tasks
Authors
VulcanBench contributors (2026)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.