report2026-09-22
Claude Opus 5.5 System Card
CoBench 2.1 contextualizes internal R&D capability through historical issue diagnosis and explicitly warns about environment drift.
AI-R&D capability
Research library
Papers & reports
Curated papers and reports on AI research capability and self-improvement.
report2026-09-03
GPT-6 Astra System Card
A task-specific AI self-improvement suite replaces older measures. Debugging, kernels, pretraining and post-training test different abilities.
AI-R&D capabilityAI-system improvement
report2026-08
Gemini 3.7 Flash Frontier Safety Framework Report
GRB broadens disclosure of internal research engineering; known task bugs and a rough historical comparator constrain interpretation.
AI-R&D capability
preprint2025-04-02
PaperBench: Evaluating AI’s Ability to Replicate AI Research
Paper replication receives partial rubric credit. Changing the agent scaffold helps some models while hurting another.
AI-R&D capabilityResearch autonomy
preprint2024-10-09
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
Competition outcomes depend on agent scaffolds, budgets and attempts. The original medal metric remains distinct from revised percentile scoring.
AI-R&D capability
report2024-11-22
Evaluating frontier AI R&D capabilities of language model agents against human experts
Agents are competitive at short resource budgets; experts gain more from longer attempts. Resource allocation matters to the comparison.
AI-R&D capability
report2026-01-29
Time Horizon 1.1
A revised task suite and evaluation platform change historical horizon estimates. Version identities prevent a false joined trend.
Research autonomyEvaluation integrity
report2025-07-10
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
Randomized AI access slowed this sample of experienced developers, illustrating why benchmark gains need real-work validation.
Observed R&D automation
report2026-02-24
We are Changing our Developer Productivity Experiment Design
Growing adoption changes who participates and which tasks are submitted. METR plans redesign because raw effects are difficult to interpret.
Evaluation integrityObserved R&D automation
report2025-06-05
Recent Frontier Models Are Reward Hacking
Examples of agents manipulating evaluation machinery show why a higher measured reward may fail to represent a better solution.
Evaluation integrity
preprint2026-08-20
AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement
Frozen repositories and hidden retraining evaluators test algorithm design. Many attempts fail to outperform the original recipe.
AI-system improvement
report2026-09-17
Measurements for understanding the pace of AI development inside frontier labs
An operational work basket distinguishes assistance, collaboration, leadership and autonomy, with approximate labor weights.
Observed R&D automation
preprint2025-05-29
Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents
Agents modify their own scaffolds and reuse descendants in further search while foundation-model weights stay fixed.
Recursive improvement evidence
report2025-05-14
AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
Evolutionary program search yields reported production improvements, including training kernels. A better target artifact is distinct from a better optimizer.
AI-system improvementRecursive improvement evidence
preprint2026-09-10
The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement
An autonomy-centered taxonomy of which improvement decisions an AI controls, what persists, and what remains externally governed.
Research autonomyRecursive improvement evidence
preprint2026-09-22
Recursive self-improvement of AI research agents
Research-agent harness evolution with held-out evaluation and a separate outer-improver comparison.
AI-system improvementRecursive improvement evidence