OSWorld
Benchmarks & Leaderboards · Free + paid
Benchmarks multimodal agents on open-ended tasks in real computer environments
At a glance
- Made by
- University of Hong Kong · Salesforce Research · CMU · University of Waterloo (joint research)
- Available on
- Web
What is OSWorld?
OSWorld is a real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across multiple operating systems (Ubuntu, Windows, macOS). On top of this environment, a 369-task benchmark was built, made up of real web and desktop apps, file input/output, and tasks spanning multiple apps. Each task has initial state setup and an execution-based evaluation script, making it reproducible. It was improved as OSWorld-Verified in 2025, and OSWorld 2.0 was released on 2026-06-26.
- 369 real computer tasks (+43 Windows)
- Supports Ubuntu, Windows, and macOS
- 134 execution-based evaluation functions
- Parallel and headless execution
- Evaluation within 1 hour with AWS support
- OSWorld-Verified·OSWorld 2.0
Who uses OSWorld and for what
- Researchers evaluate the performance of computer-use agents (LLM/VLM)
Pricing
Free (public benchmark/leaderboard)
Mostly free leaderboards/datasets released for academic and community purposes
Estimate based on AI knowledge - please verify on the official site
AI tools similar to OSWorld (alternatives)
OSWorld vs. alternatives at a glance
| AI | Pricing | Combos on the map | Connected tools | Free | Cheapest monthly | Korean |
|---|---|---|---|---|---|---|
| OSWorld | Free + paid | 0 | 5 | — | — | — |
| APEX-Agents | Free + paid | 0 | 11 | — | — | No |
| ARC-AGI-2 | Free + paid | 0 | 11 | — | — | No |
| Context-Bench | Free + paid | 0 | 11 | — | — | No |
| MCP Atlas | Free + paid | 0 | 11 | — | — | No |
0 of these 5 tools have a free plan.
Combo and connection counts are based on the relationships recorded on AI Galaxy Map (2026-10-26); they are not a quality ranking.
OSWorld on AI Galaxy Map
- One of 27 AI tools in the Benchmarks & Leaderboards field.
- Connected to 5 other tools — #11 in its field.
Frequently asked questions
Can I use OSWorld for free?
Free (public benchmark/leaderboard). Prices can change, so check the official site before paying.
What AI can I use instead of OSWorld?
Tools that do the same job, such as APEX-Agents, ARC-AGI-2, Context-Bench, MCP Atlas, are connected to it on the map.
Sources (official pages)
Checked on the official pages on 2026-10-14. Prices and features can change, so check the official site again before paying.