Tracker

benchmark

Task-completion time horizons

Human-task duration associated with 50% agent success.

Research autonomy

What this measure tells us

Supporting measure of research and software-task autonomy.

  • Human task duration is not uninterrupted AI runtime.
  • Task distribution and scaffold revisions change estimates.

Results by version

TH1.1

2026-01-29 · 228 tasks

Appendix estimates published together on January 29; evaluation dates not supplied.

Task-completion time horizons · 50% task-completion horizon

limited comparison

Configurations and budgets not fully captured in this extraction. No automatic deltas or joined calendar trend.

Task-completion time horizons: 50% task-completion horizonCategorical point plot with no connecting line. Source explicitly reports these systems under the same evaluation design.0200400minutes2026-01-29: 214 minutes · GPT-5 / METR horizon evaluation12142026-01-29: 121 minutes · o3 / METR horizon evaluation21212026-01-29: 101 minutes · Claude Opus 4 / METR horizon evaluation31012026-01-29: 320 minutes · Claude Opus 4.5 / METR horizon evaluation43202026-01-29: 60 minutes · Claude 3.7 Sonnet / METR horizon evaluation560
Published results · TH1.1
KeySystem / organizationMetricReported resultDateProtocolVerificationEvidence
4Claude Opus 4.5 / METR horizon evaluation50% task-completion horizon320 minutes2026-01-29th-1-1-pindependent evaluation
1GPT-5 / METR horizon evaluation50% task-completion horizon214 minutes2026-01-29th-1-1-pindependent evaluation
3Claude Opus 4 / METR horizon evaluation50% task-completion horizon101 minutes2026-01-29th-1-1-pindependent evaluation
2o3 / METR horizon evaluation50% task-completion horizon121 minutes2026-01-29th-1-1-pindependent evaluation
5Claude 3.7 Sonnet / METR horizon evaluation50% task-completion horizon60 minutes2026-01-29th-1-1-pindependent evaluation

TH1 (historical estimates)

Release date not reported · 170 tasks

Appendix estimates published together on January 29; evaluation dates not supplied.

Task-completion time horizons · 50% task-completion horizon

limited comparison

Configurations and budgets not fully captured in this extraction. No automatic deltas or joined calendar trend.

Task-completion time horizons: 50% task-completion horizonCategorical point plot with no connecting line. Source explicitly reports these systems under the same evaluation design.0150300minutes2026-01-29: 138 minutes · GPT-5 / METR horizon evaluation11382026-01-29: 94 minutes · o3 / METR horizon evaluation2942026-01-29: 86 minutes · Claude Opus 4 / METR horizon evaluation3862026-01-29: 289 minutes · Claude Opus 4.5 / METR horizon evaluation42892026-01-29: 56 minutes · Claude 3.7 Sonnet / METR horizon evaluation556
Published results · TH1 (historical estimates)
KeySystem / organizationMetricReported resultDateProtocolVerificationEvidence
4Claude Opus 4.5 / METR horizon evaluation50% task-completion horizon289 minutes2026-01-29th-1-0-pindependent evaluation
1GPT-5 / METR horizon evaluation50% task-completion horizon138 minutes2026-01-29th-1-0-pindependent evaluation
3Claude Opus 4 / METR horizon evaluation50% task-completion horizon86 minutes2026-01-29th-1-0-pindependent evaluation
2o3 / METR horizon evaluation50% task-completion horizon94 minutes2026-01-29th-1-0-pindependent evaluation
5Claude 3.7 Sonnet / METR horizon evaluation50% task-completion horizon56 minutes2026-01-29th-1-0-pindependent evaluation

Version lineage

TH1 (historical estimates) ← TH1.1TH1 (historical estimates) → TH1.1

Reference points

No applicable reference points are published for this measure.

Availability

What is publicly available
ResourceStatus
public descriptionyes
public resultsyes
public tasksyes
public codeyes
public evaluation serviceunknown

Official sources