Note: Overall leaderboard rankings may not reflect true model quality — individual benchmarks give a clearer picture. ARC-Challenge MMLU GPQA GSM8K Artificial Analysis Intelligence Index v4.1
← Back to leaderboard

SWE-bench Pro

69 models

SWE-bench Pro — a benchmark by Scale AI (Xiang Deng, Jeff Da et al., arXiv 2509.16941). Contains 1,865 software engineering tasks from 41 actively maintained repositories (Python, Go, TypeScript, JavaScript). The agent must fix real bugs and implement features — multi-file patches with substantial code changes. Tasks take a professional engineer hours to days to complete. All tasks are human-verified. Score = % of issues resolved correctly.

Top 5 Models Performance

Anthropic: claude-opus-5.5 ######################################## 89.9
Anthropic: Claude Fable 5 #################################### 80.3
Ornith: ornith-1.5-122b ################################## 76.9
sakana: fugu-ultra ################################# 73.7
Anthropic: Claude Opus 4.8 ############################### 69.2
– 970 – 1.1T
– 2019 – 2026
Rank Model Score
🇺🇸 1 Anthropic: claude-opus-5.5 89.9
🇺🇸 2 Anthropic: Claude Fable 5 80.3
❓ 3 Ornith: ornith-1.5-122b 76.9
🇯🇵 4 sakana: fugu-ultra 73.7
🇺🇸 5 Anthropic: Claude Opus 4.8 69.2
🇨🇳 6 inclusionAI: ling-3.1-flash 68.74
🇨🇳 7 Qwen: qwen3.8-2.4t-a95b通义千问 67.7
❓ 8 reflection: beam-1.0 66.1
🇨🇳 9 Tencent: hy4-preview腾讯控股有限公司 65.7
❓ 10 Ornith: ornith-1.5-397b 65.1
🇺🇸 11 Anthropic: Claude Opus 4.7 64.3
🇺🇸 12 Anthropic: claude-sonnet-5 63.2
🇨🇳 13 Qwen: qwen3.8-flash-next通义千问 62.5
❓ 14 Ornith: ornith-1.0-397b 62.2
🇨🇳 15 Z.ai: GLM 5.2北京智谱华章科技有限公司 62.1
🇨🇳 16 Qwen: qwen3.8-27b通义千问 61.7
🇺🇸 17 Meta: muse-spark-1.1 61.5
❓ 18 recursiveai: ripple-65b-preview 61.42
❓ 19 internscience: sage-a1.5 61.2
❓ 20 nanbeige: nanbeige4.3-33b 60.3
❓ 21 Ornith: ornith-1.5-35b-a3b 59.6
🇨🇳 22 MiniMax: MiniMax M3稀宇科技 59
🇯🇵 23 sakana: fugu 59
🇨🇳 24 Nex AGI: Nex-N2 Pro 58.8
🇨🇳 25 Moonshot AI: Kimi K2.6月之暗面 58.6
🇨🇳 26 Z.ai: GLM 5.1北京智谱华章科技有限公司 58.4
🇨🇳 27 Tencent: hy3腾讯控股有限公司 57.9
🇨🇳 28 Qwen: qwen3.7-plus通义千问 57.6
🇨🇳 29 Xiaomi: MiMo-V2.5 Pro小米集团 57.2
🇺🇸 30 Anthropic: claude-opus-4.5 57.1
🇨🇳 31 inclusionAI: ling-3.0-flash 56.6
🇨🇳 32 StepFun: step-3.7-flash上海阶跃星辰智能科技有限公司 56.3
🇨🇳 33 MiniMax: minimax-m2.7稀宇科技 56.2
🇨🇳 34 Xiaomi: MiMo-V2.5小米集团 56.1
🇨🇳 35 DeepSeek: deepseek-v4-pro-0813深度求索 55.4
🇨🇳 36 MiniMax: MiniMax M2.5稀宇科技 55.4
🇨🇳 37 DeepSeek: DeepSeek V4 Pro深度求索 55.4
🇺🇸 38 Meta: Muse Spark 55
❓ 39 xhtoken: spark-x2.6-4b 54.3
🇺🇸 40 Google: Gemini 3.1 Pro Preview 54.2
❓ 41 Orion LLM Labs: grm-2.6-plus 54
🇨🇳 42 Qwen: qwen3.6-27b通义千问 53.5
🇺🇸 43 NVIDIA Corporation: nvidia-nemotron-3.5-super-vl-120b-a12b 51.78
🇺🇸 44 Meta: muse-glimmer-30b 51.2
🇨🇳 45 Qwen: Qwen3.5 27B通义千问 51.2
🇨🇳 46 Moonshot AI: Kimi K2.5月之暗面 50.7
❓ 47 Ornith: ornith-1.0-35b 50.4
🇨🇳 48 Nex AGI: Nex-N2 Mini 50.2
🇨🇳 49 Qwen: qwen3.6-35b-a3b通义千问 49.5
❓ 50 Ornith: ornith-1.5-9b 47.5
❓ 51 diffuspace: clover-35b-a3b-preview 46.5
🇺🇸 52 Poolside: laguna-xs.2 46.3
❓ 53 muveraai: laguna-xs.2 44.5
❓ 54 xhtoken: spark-x2.5-4b 44.4
🇨🇳 55 Qwen: qwen3-coder-next通义千问 44.3
❓ 56 Ornith: ornith-1.0-9b 42.9
🇨🇦 57 Cohere Labs: north-mini-code-1.0 40.2
🇨🇳 58 Qwen: qwen3-coder-480b-a35b-instruct通义千问 38.7
🇨🇳 59 MiniMax: MiniMax M2.1稀宇科技 36.81
🇨🇳 60 Moonshot AI: Kimi K2 Instruct月之暗面 27.67
🇨🇳 61 Qwen: qwen3-235b-a22b通义千问 21.41
🇺🇸 62 OpenAI: gpt-oss-120b 16.2
🇨🇳 63 DeepSeek: deepseek-v3.2深度求索 15.56
🇨🇳 64 OpenBMB: minicpm5-2b 14.4
🇺🇸 65 Google: Gemma 3 27B Instruct 11.38
🇺🇸 66 Meta Llama: Llama 3.1 405B Instruct 11.18
❓ 67 xhtoken: spark-x2.5-1.7b 10.4
🇨🇳 68 Z.ai: glm-4.6北京智谱华章科技有限公司 9.67
🇺🇸 69 Meta Llama: Llama 4 Maverick Instruct 5.24