preprint · 2024-10-09
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Lilian Weng, Aleksander Mądry
Why it matters here
Competition outcomes depend on agent scaffolds, budgets and attempts. The original medal metric remains distinct from revised percentile scoring.
AI-R&D capability
What to keep in mind
- Original v1 snapshot; later paper revisions exist and are not silently substituted.