SWE-bench
Benchmarks & Leaderboards · Free + paid
Evaluates LLM performance on real-world software issues collected from GitHub
At a glance
- Made by
- SWE-bench Team
- Korean
- None (English and a few others only)
What is SWE-bench?
SWE-bench is the original benchmark, built from 2,294 real GitHub issues across 12 Python repositories, and each entry is reported as the percentage of tasks resolved (% Resolved). There are several leaderboards, including Verified (500 human-filtered tasks), Lite (300 tasks for low-cost evaluation), Multilingual (300 tasks from 42 repositories in 9 programming languages), Multimodal (480 tasks with visual elements), and Bash Only, which compares models in the same mini-SWE-agent environment. Results can be compared with various charts, such as resolve rate versus cost.
- Original SWE-bench (2,294 tasks)
- SWE-bench Verified (500 human-reviewed tasks)
- Lite (300 tasks), Multilingual (300 tasks, 9 languages), Multimodal (480 tasks)
- Bash Only: compared in the same mini-SWE-agent environment
- Result comparison charts for resolve rate, cost, number of steps, and more
- Filter for selecting open-weight models
- Related tools such as SWE-agent, SWE-smith, and the SWE-bench CLI
Pricing
Free (public benchmark/leaderboard)
Mostly free leaderboards/datasets released for academic and community purposes
Estimate based on AI knowledge - please verify on the official site
AI tools similar to SWE-bench (alternatives)
SWE-bench vs. alternatives at a glance
| AI | Pricing | Combos on the map | Connected tools | Free | Cheapest monthly | Korean |
|---|---|---|---|---|---|---|
| SWE-bench | Free + paid | 0 | 5 | — | — | No |
| Aider Leaderboard | Free + paid | 0 | 12 | — | — | No |
| CORE-Bench Hard | Free + paid | 0 | 12 | — | — | No |
| Live-SWE-agent | Free + paid | 0 | 12 | — | — | No |
0 of these 4 tools have a free plan.
Combo and connection counts are based on the relationships recorded on AI Galaxy Map (2026-10-26); they are not a quality ranking.
SWE-bench on AI Galaxy Map
- One of 27 AI tools in the Benchmarks & Leaderboards field.
- Connected to 5 other tools — #11 in its field.
Frequently asked questions
Can I use SWE-bench for free?
Free (public benchmark/leaderboard). Prices can change, so check the official site before paying.
What AI can I use instead of SWE-bench?
Tools that do the same job, such as Aider Leaderboard, CORE-Bench Hard, Live-SWE-agent, BigCodeArena, are connected to it on the map.
Can I use SWE-bench in Korean?
None (English and a few others only)
Sources (official pages)
Checked on the official pages on 2026-10-14. Prices and features can change, so check the official site again before paying.