Research archive / Development through Hard v1

Explore the
saved runs.

Development through Hard v1

Browse saved runs, prompts and model answers.

16K vs. 32K token study → This separate study is not included in the totals below.

Triangle confirmation · 17 Nov 2026 → 576/576 responses on 96 fresh graphs. Reported separately from the archive totals below.

Saved run groups8Plus the earlier reported paper study
Evaluation requests5,616Includes API failures; counts reused requests once
Reused responses48Claude responses from the earlier replication
Requests without a response2Step requests that failed in the pilot

Compare results within each run; datasets and settings differ.

Archive coverage and reused results
Compare models within the same run. Tasks, seeds, token budgets and providers differ between runs. Each row shows the settings used. The pilot reuses 48 Claude responses from the replication; the request total counts them once. Downloads retain all 5,690 attempts, including 76 failed attempts and 74 retries. The explorer shows the selected saved response for each question.

Opus across settings

The 24/24 family-pilot row is a subset of the larger replication.

Show every Opus run →
Development · 2,048 tokens / low3 errors28/48 correct · 17 cutoffs
Two holdouts · 4,096 tokens / high7 cutoffs13/20 correct · 0 completed errors
Full replication · 8,192 tokens / high5 cutoffs91/96 correct · 0 completed errors
Claude responses reused in the pilot24/246 graphs from the 24-graph replication
Loading runs…
Earlier reported study

Results from the paper

These scores come from the paper. The original model responses are unavailable.