Skip to content

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

A continuously updated coding benchmark built from newly collected LeetCode, AtCoder, and Codeforces problems. Fresh problem windows reduce one contamination path, but results still need a release and setup check.

10 models

Higher is better.

  1. 1Solar Pro 4Upstage87.8
  2. 2Grok 3 MinixAI74.8
  3. 3DeepSeek V3.1 (Reasoning)DeepSeek74.8
  4. 4Nemotron Ultra 253BNVIDIA66.3
  5. 5MiniMax M1 80kMiniMax65.0
  6. 6Grok 3 [Beta]xAI57.0
  7. 7DeepSeek V3.1DeepSeek56.4
  8. 8Phi-4-reasoningMicrosoft53.8
  9. 9Qwen2.5-MaxAlibaba38.7
  10. 10DeepSeek V3DeepSeek37.6
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Competitive-programming evaluation
Tasks
Continuously updated contest problems
Authors
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, Ion Stoica (2024)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.