CORE-Bench Hard
Benchmarks & Leaderboards · Free + paid
Agent installs deps, runs published scientific code, and answers questions
At a glance
- Made by
- HAL
- Korean
- None (English and a few others only)
What is CORE-Bench Hard?
CORE-Bench evaluates an agent's ability to computationally reproduce the results of published scientific papers. At the Hard level, only the paper's codebase is provided, and the agent must install libraries and dependencies, run the code, read the outputs and figures, and answer questions about the paper. It consists of 90 papers (45 in the public test set), and 49 runs were evaluated across 3 scaffolds and 27 models. The HAL leaderboard has currently stopped updating for new models and is focusing on measuring agent reliability.
- Tasks based on real published papers
- Evaluates code understanding, dependency management, and result interpretation together
- 3 difficulty levels from easy to hard
- 90 papers, 45 in the public test set
- 49 runs evaluated (3 scaffolds, 27 models)
- Shows accuracy and cost by model
Pricing
Free (public benchmark/leaderboard)
Mostly free leaderboards/datasets released for academic and community purposes
Estimate based on AI knowledge - please verify on the official site
AI tools similar to CORE-Bench Hard (alternatives)
CORE-Bench Hard vs. alternatives at a glance
| AI | Pricing | Combos on the map | Connected tools | Free | Cheapest monthly | Korean |
|---|---|---|---|---|---|---|
| CORE-Bench Hard | Free + paid | 0 | 12 | — | — | No |
| Aider Leaderboard | Free + paid | 0 | 12 | — | — | No |
| Live-SWE-agent | Free + paid | 0 | 12 | — | — | No |
| SWE-bench | Free + paid | 0 | 5 | — | — | No |
| SWE-bench Multilingual | Free + paid | 0 | 5 | — | — | No |
0 of these 5 tools have a free plan.
Combo and connection counts are based on the relationships recorded on AI Galaxy Map (2026-10-26); they are not a quality ranking.
CORE-Bench Hard on AI Galaxy Map
- One of 27 AI tools in the Benchmarks & Leaderboards field.
- Connected to 12 other tools — #1 in its field.
Frequently asked questions
Can I use CORE-Bench Hard for free?
Free (public benchmark/leaderboard). Prices can change, so check the official site before paying.
What AI can I use instead of CORE-Bench Hard?
Tools that do the same job, such as Aider Leaderboard, Live-SWE-agent, SWE-bench, SWE-bench Multilingual, are connected to it on the map.
Can I use CORE-Bench Hard in Korean?
None (English and a few others only)
Sources (official pages)
Checked on the official pages on 2026-10-14. Prices and features can change, so check the official site again before paying.