SWE-rebench
Benchmarks & Leaderboards · Free + paid
Continuously evolving and decontaminated benchmark for software engineering LLMs
At a glance
- Korean
- None (English and a few others only)
- Available on
- Web
What is SWE-rebench?
SWE-rebench is a benchmark created to address SWE-bench's problems of data contamination, non-comparable results due to scaffolding differences, and insufficient verification. It aims to continuously re-evaluate LLMs in a standardized way on real software tasks, separating the LLM's own contribution from overall agent performance. The leaderboard lets you choose a period and shows resolve rate, Pass@5, and cost and tokens per problem for the problems in that period (e.g., 111 problems from 65 repositories), and it also flags possible contamination. It compares not only models but also agents such as Claude Code, Codex, and Cursor.
- Leaderboard with selectable period (time window)
- Shows resolve rate, Pass@5, and cost and tokens per problem
- Potential contamination flag
- Models and agents compared together
- SWE-rebench v2 leaderboard
- New models added regularly
Who uses SWE-rebench and for what
- Model developers and researchers fairly compare LLM coding ability
Pricing
Free (public benchmark/leaderboard)
Mostly free leaderboards/datasets released for academic and community purposes
Estimate based on AI knowledge - please verify on the official site
AI tools similar to SWE-rebench (alternatives)
SWE-rebench vs. alternatives at a glance
| AI | Pricing | Combos on the map | Connected tools | Free | Cheapest monthly | Korean |
|---|---|---|---|---|---|---|
| SWE-rebench | Free + paid | 0 | 5 | — | — | No |
| Aider Leaderboard | Free + paid | 0 | 12 | — | — | No |
| CORE-Bench Hard | Free + paid | 0 | 12 | — | — | No |
| Live-SWE-agent | Free + paid | 0 | 12 | — | — | No |
0 of these 4 tools have a free plan.
Combo and connection counts are based on the relationships recorded on AI Galaxy Map (2026-10-26); they are not a quality ranking.
SWE-rebench on AI Galaxy Map
- One of 27 AI tools in the Benchmarks & Leaderboards field.
- Connected to 5 other tools — #11 in its field.
Frequently asked questions
Can I use SWE-rebench for free?
Free (public benchmark/leaderboard). Prices can change, so check the official site before paying.
What AI can I use instead of SWE-rebench?
Tools that do the same job, such as Aider Leaderboard, CORE-Bench Hard, Live-SWE-agent, BigCodeArena, are connected to it on the map.
Can I use SWE-rebench in Korean?
None (English and a few others only)
Sources (official pages)
Checked on the official pages on 2026-10-14. Prices and features can change, so check the official site again before paying.