SWE-DEV
Benchmarks & Leaderboards · Free + paid
Evaluating and training autonomous feature-driven software development agents
At a glance
- Available on
- CLI
- Open source
- Apache 2.0
- GitHub
- 62 stars
What is SWE-DEV?
SWE-Dev is a benchmark and training corpus for feature-driven development (FDD) tasks, providing 14,000 training tasks and 500 test tasks. Each task includes an executable environment and developer-written unit tests, so it can be used for supervised fine-tuning and reinforcement learning with execution rewards. Each task bundles source code, dependencies, a Dockerfile, and tests so it can be reproduced, and it gives a reward signal based on pytest pass/fail. It also includes single-agent and MASLab-based multi-agent inference scripts.
- 14,000 training tasks and 500 test tasks
- Includes executable Docker environments and unit tests
- Pytest-based reward signal for reinforcement learning
- Easy/Hard difficulty split
- Evaluation API server for reinforcement learning
- Single-agent and multi-agent inference scripts (10 MASLab types)
Who uses SWE-DEV and for what
- Researchers train coding models with fine-tuning and reinforcement learning
- Evaluates a model's ability to develop new features
Pricing
Free (public benchmark/leaderboard)
Mostly free leaderboards/datasets released for academic and community purposes
Estimate based on AI knowledge - please verify on the official site
AI tools similar to SWE-DEV (alternatives)
SWE-DEV vs. alternatives at a glance
| AI | Pricing | Combos on the map | Connected tools | Free | Cheapest monthly | Korean |
|---|---|---|---|---|---|---|
| SWE-DEV | Free + paid | 0 | 5 | — | — | — |
| Aider Leaderboard | Free + paid | 0 | 12 | — | — | No |
| CORE-Bench Hard | Free + paid | 0 | 12 | — | — | No |
| Live-SWE-agent | Free + paid | 0 | 12 | — | — | No |
0 of these 4 tools have a free plan.
Combo and connection counts are based on the relationships recorded on AI Galaxy Map (2026-10-26); they are not a quality ranking.
SWE-DEV on AI Galaxy Map
- One of 27 AI tools in the Benchmarks & Leaderboards field.
- Connected to 5 other tools — #11 in its field.
Frequently asked questions
Can I use SWE-DEV for free?
Free (public benchmark/leaderboard). Prices can change, so check the official site before paying.
What AI can I use instead of SWE-DEV?
Tools that do the same job, such as Aider Leaderboard, CORE-Bench Hard, Live-SWE-agent, BigCodeArena, are connected to it on the map.
Sources (official pages)
Checked on the official pages on 2026-10-14. Prices and features can change, so check the official site again before paying.