CANONICAL BENCHMARK • EVALUATION SPECIFICATION

Terminal-Bench 2.0

Autonomous CLI agent tasks measuring terminal command generation, execution, and multi-step system navigation.

What Terminal-Bench 2.0 Actually Measures

Autonomous command-line execution, shell tool use, multi-step debugging, environment triage, and terminal state evaluation in isolated Linux sandboxes.

⚠ Known Boundaries & Harness Sensitivity

Does not measure long-context codebase refactoring; highly sensitive to system prompt formatting and sub-shell execution timeouts.

Complete Verified Leaderboard

Fully server-side rendered table with verified primary source attributions and direct repository links.

Page updated: 89 Evaluated Tasks • Metric: Task Accuracy
SOURCE & HARNESS
#01 Claude 3.7 Sonnet 58.4% $0.00 24 Feb 2025 Terminal-Bench Submissions →
MIT reported
#02 o3-mini (High) 43.8% $0.00 5 Feb 2025 Terminal-Bench Submissions →
MIT reported
#03 Claude 3.5 Sonnet (20241022) 42.1% $0.00 25 Oct 2024 Terminal-Bench Submissions →
MIT reported
#04 DeepSeek R1 39.3% $0.00 28 Jan 2025 Terminal-Bench Submissions →
MIT reported
Sources & Open Data Licences CC-BY / Open Source

In compliance with upstream dataset licenses, all evaluated telemetry rows are credited to their primary sources:

What This Benchmark Does Not Measure

Does not measure long-context codebase refactoring or GUI interaction.

How to Reproduce This Evaluation

All evaluations published by Vibecoder Journal follow publicly documented harness specifications. You can execute this test suite against any local or API model: 

# Standardized containerized evaluation execution
git clone https://tbench.ai/ && cd evaluation
docker build -t vcj-harness .
docker run --rm -v $(pwd)/results:/results vcj-harness --benchmark terminal_bench
Sources & Open Data Licences CC-BY / Open Source

In compliance with upstream dataset licenses, all evaluated telemetry rows are credited to their primary sources:

Abdullah Zulfiqar
Abdullah Zulfiqar Founder & Technical Editor

Abdullah Zulfiqar is the founder and editor of Vibe Coder Journal, an independent publication that benchmarks AI coding tools. He verifies every figure against primary sources — official documentation, real release files and live leaderboards — rather than repeating secondary reporting. His work has corrected widely-circulated errors in Terminal-Bench scores, Ollama's official uninstall instructions and Anthropic's documented install commands. Vibe Coder Journal accepts no sponsorships or affiliate commissions.

All Benchmarks → LinkedIn Profile →

Related AI Coding Benchmarks

benchmark 2 min read

LiveCodeBench

Contamination-free competitive programming problems from LeetCode, Codeforces, and AtCoder updated continuously.

By Abdullah Zulfiqar • Sep 4, 2026
benchmark 3 min read

Aider Polyglot Benchmark

Multi-language code editing benchmark evaluating diff formatting and test suite pass rates across Python, JS, Go, Rust, C++, and Java.

By Abdullah Zulfiqar • Sep 4, 2026