Cybench

A cybersecurity benchmark of professional Capture the Flag tasks for measuring autonomous cyber agent capability and risk.

2 models

Higher is better.

Higher is better. Bar colour: blue proprietary, orange open weights, green other.

Metric: Cybersecurity agent task completion
Tasks: 40 professional CTF tasks
Authors: Andy K. Zhang, Neil Perry, Riya Dulepet, Joey Ji, Celeste Menders, Justin W. Lin, Eliot Jones, Gashon Hussein, Samantha Liu, Donovan Jasper, Pura Peetathawatchai, Ari Glenn, Vikram Sivashankar, Daniel Zamoshchin, Leo Glikbarg, Derek Askaryar, Mike Yang, Teddy Zhang, Rishi Alluri, Nathan Tran, Rinnara Sangpisit, Polycarpos Yiorkadjis, Kenny Osele, Gautham Raghupathi, Dan Boneh, Daniel E. Ho, Percy Liang (2025)
Source: Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models ↗

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.