Our Mission
Vibecoder Journal is an independent technical publication dedicated to rigorous, empirical evaluations of Large Language Models (LLMs) and autonomous AI coding agents. Our coverage includes published benchmark results, practical setup guides and tool comparisons. Each evaluation should identify whether its evidence comes from our own run or an external report; a model profile alone is not evidence of a new independent test.
Benchmarking Methodology & Composite Scoring
The leaderboard has two distinct views. Published Index shows the stored editorial scores and ranks. Those stored values are not recalculated by moving the sliders, and the underlying aggregation should not be inferred from the custom calculator.
Custom Score is an exploratory weighted arithmetic mean of three recorded benchmark percentages: GPQA Diamond, SWE-bench Verified and Terminal-Bench. Slider weights are normalized to a total of 100%. For example, weights 45, 40 and 15 produce (45 × GPQA + 40 × SWE + 15 × Terminal) / 100. A model missing any required benchmark is left unscored; missing results are not replaced with synthetic values. Selecting Reset Published View restores the stored index.
- GPQA Diamond: graduate-level scientific question answering, rather than a direct coding test.
- SWE-bench Verified: repository issue resolution; scores depend on the agent, harness and evaluation settings.
- Terminal-Bench: tasks performed in a terminal environment; compare results only with matching benchmark versions and setups.
The $/point column divides the displayed blended price per million tokens by the displayed score. It is a comparison aid, not the total cost of completing a task. Prices, token mixes, retries and caching affect actual bills.
Benchmark results from different vendors or harnesses may not be directly comparable. Source links, test dates, model versions, prompts and runnable artifacts are needed to reproduce a particular evaluation. Where those details are unavailable, treat the result as a reported snapshot rather than an independently reproduced measurement.
Editorial Independence & Ethics
All benchmark datasets, review hardware, and inference API calls are acquired independently. We do not accept sponsored placements that alter ranking positions. When affiliate links or commercial relations exist for tools, they are explicitly disclosed in accordance with FTC guidelines and our internal ethics policy.
Editorial Leadership
Editorial Standards & Contact Desk
We welcome corrections, benchmark reproduction submissions, and technical feedback from researchers and developers:
- Editorial Desk & Tips: abdullah@vibecoderjournal.com
- Benchmark Correction Submissions: abdullah@vibecoderjournal.com
- Founder Direct: abdullah@vibecoderjournal.com