report · 2026-02-24
We are Changing our Developer Productivity Experiment Design
Joel Becker, Nate Rush, Tom Cunningham, David Rein, Khalid Mahamud
Why it matters here
Growing adoption changes who participates and which tasks are submitted. METR plans redesign because raw effects are difficult to interpret.
Evaluation integrityObserved R&D automation
What to keep in mind
- Cannot infer a reliable current uplift from these selected participants.