MCP Atlas
Benchmarks & Leaderboards · Free + paid
Evaluates how well language models handle real-world tool use through MCP
At a glance
- Made by
- Scale AI
- Korean
- None (English and a few others only)
- Available on
- Web
- Open source
- Open source (license name not stated)
What is MCP Atlas?
MCP-Atlas evaluates how well language models use real tools through MCP. It has 1,000 human-written tasks (500 public, 500 private) spanning 36 real MCP servers and 220 tools, and each task requires 3 to 6 tool calls. The model must find the right tools in a list mixed with distractor tools, call them with the correct arguments, and recover from errors to produce a final answer. The correct answer is split into claims and graded by an LLM judge, and coverage of 75% or more counts as a pass.
- 1,000 tasks (500 public, 500 private)
- 36 real MCP servers and 220 tools (Docker)
- Tool exposure including distractor tools
- Claim-level LLM judge grading and pass rate metric
- Budget of up to 100 tool calls per task
- Paper, HuggingFace dataset, and GitHub harness released
Who uses MCP Atlas and for what
- Model developers compare tool-use agent performance
Pricing
Free (public benchmark/leaderboard)
Mostly free leaderboards/datasets released for academic and community purposes
Estimate based on AI knowledge - please verify on the official site
AI tools similar to MCP Atlas (alternatives)
MCP Atlas vs. alternatives at a glance
| AI | Pricing | Combos on the map | Connected tools | Free | Cheapest monthly | Korean |
|---|---|---|---|---|---|---|
| MCP Atlas | Free + paid | 0 | 11 | — | — | No |
| APEX-Agents | Free + paid | 0 | 11 | — | — | No |
| ARC-AGI-2 | Free + paid | 0 | 11 | — | — | No |
| Context-Bench | Free + paid | 0 | 11 | — | — | No |
| OSWorld | Free + paid | 0 | 5 | — | — | — |
0 of these 5 tools have a free plan.
Combo and connection counts are based on the relationships recorded on AI Galaxy Map (2026-10-26); they are not a quality ranking.
MCP Atlas on AI Galaxy Map
- One of 27 AI tools in the Benchmarks & Leaderboards field.
- Connected to 11 other tools — #6 in its field.
Frequently asked questions
Can I use MCP Atlas for free?
Free (public benchmark/leaderboard). Prices can change, so check the official site before paying.
What AI can I use instead of MCP Atlas?
Tools that do the same job, such as APEX-Agents, ARC-AGI-2, Context-Bench, OSWorld, are connected to it on the map.
Can I use MCP Atlas in Korean?
None (English and a few others only)
Sources (official pages)
Checked on the official pages on 2026-10-14. Prices and features can change, so check the official site again before paying.