swyxbench An open research archive for AI engineering Source & methods ↗

Working papers · reproducible systems research

Benchmarks deserve
the whole paper.

Questions, source evidence, experimental design, measured results, charts, failure cases, and limitations—published together so an engineering claim can be inspected and reproduced.

Read the current reports ↓

Current working papers

Research reports

Each paper has a checked receipt, a source-linked figure, a reproducible protocol, and an explicit evidence boundary.

Editorial policy

A result without its caveats is not a result.

Evidence before narrative

Source versions, model revisions, checksums, and measurement conditions accompany the claim.

Figures must answer a question

Charts show observed comparable measurements; missing human gold is never replaced by invented accuracy.

Limitations stay visible

Incomplete datasets, provisional labels, missing transcripts, and unsupported conclusions appear beside the results.