Skip to content

OSWorld-Verified

OSWorld-Verified is the July 2025 repaired release of OSWorld's real-computer evaluation. It measures whether a model-agent system can finish desktop and web tasks from configured starting states, with success checked by execution-based evaluators.

35 models

Higher is better.

  1. 1Qwen3.8 MaxAlibaba86.1
  2. 2Claude Fable 5Anthropic85.0
  3. 3Claude Mythos 5Anthropic85.0
  4. 4Qwen3.8-27BAlibaba84.3
  5. 5Claude Opus 4.8Anthropic83.4
  6. 6Gemini 3.6 FlashGoogle83.0
  7. 7MiMo-V2.6-ProXiaomi82.0
  8. 8Claude Sonnet 5Anthropic81.2
  9. 9Muse Spark 1.1Meta80.8
  10. 10MiMo-V2.6-FlashXiaomi80.8
  11. 11Holo3-122B-A10BH Company78.8
  12. 12GPT-5.5OpenAI78.7
  13. 13Gemini 3.5 FlashGoogle78.4
  14. 14Claude Opus 4.7 (Adaptive)Anthropic78.0
  15. 15UI-Mate-27BTencent77.0
  16. 16GPT-5.4OpenAI75.0
  17. 17Gemini 3.5 Flash-LiteGoogle74.0
  18. 18Qwen3.7 PlusAlibaba73.3
  19. 19Kimi K2.6Moonshot AI73.1
  20. 20Claude Opus 4.6Anthropic72.7
  21. 21Claude Sonnet 4.6Anthropic72.1
  22. 22GPT-5.4 miniOpenAI72.1
  23. 23MiniMax M3MiniMax70.1
  24. 24Claude Opus 4.5Anthropic66.3
  25. 25Muse Glimmer 30BMeta65.9
  26. 26GPT-5.3 CodexOpenAI64.7
  27. 27Kimi K2.5 (Reasoning)Moonshot AI63.3
  28. 28Kimi K2.5Moonshot AI63.3
  29. 29Qwen3.5 397BAlibaba62.2
  30. 30Claude Sonnet 4.5Anthropic61.4
  31. 31Qwen3.5-122B-A10BAlibaba58.0
  32. 32Qwen3.5-27BAlibaba56.2
  33. 33Qwen3.5-35B-A3BAlibaba54.5
  34. 34GPT-5.2OpenAI47.3
  35. 35GPT-5.4 nanoOpenAI39.0
Higher is better. Bar colour: blue proprietary, orange open weights, green other.

About this benchmark

Metric
Execution-based interactive task success
Tasks
369 real-world computer tasks (361 when eight Google Drive tasks are excluded)
Authors
Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, Tao Yu (2025)

Scores are as reported by model developers or evaluators and may use different settings (for example reasoning effort or tools). Compare with care.