benchmark

GRB internal research engineering

End-to-end research engineering in an internal coding scaffold.

AI-R&D capability

What this measure tells us

Tests task chaining relevant to research automation.

Benchmark version

Metric

Results to compare

Protocol

2026-08 · 74 tasks

Task set includes known buggy tasks.

limited comparison

Rough older estimate; no automatic delta.

Published results · August 2026 report snapshot
Key	System / organization	Metric	Reported result	Date	Protocol	Verification	Evidence
1	Gemini 3.7 Flash / GRB scaffold	Average pass@1	27%	2026-08	grb-current	lab reported	Gemini 3.7 Flash Frontier Safety Framework Report ↗
2	Gemini 3.1 Pro / preliminary GRB runs	Average pass@1	around 16%	2026-08	grb-earlier	lab reported	Gemini 3.7 Flash Frontier Safety Framework Report ↗

policy threshold

DeepMind uses 90% as a conservative CCL rule-out boundary, not proof that a system reaching it automates research.

What this reference means: policy trigger · Where it applies: established