NeFut Logo NeFut
中 Admin Login

[CS.AI] ReproBridge: Embedding Reproducibility into ML Research Artifacts

Published at: 2026-11-07 22:00 Last updated: 2026-11-08 12:04
#AI #Machine Learning #Open Source

Reproducing machine learning experiments often requires piecing together procedures from papers, released code, and supporting artifacts. For coding agents, success depends more on their ability to fill missing details than on the released code alone. We investigate whether preserving recovered procedures and decisions improves subsequent independent reproduction.\ \ We define a standardized artifact design consisting of a refined codebase, environment setup, experiment launchers, and a reproduction guide, and build the ReproBridge workflow to construct such reproducible artifacts. ReproBridge‑Bench contains 1,029 tasks derived from 387 experiments across 36 papers, each anchored by five human‑verified reference scores. Agents attempt tasks in fresh sessions using either raw code or refined artifacts, while a separate Auditor checks execution evidence.\ \ Results show that refined artifacts raise mean reproduction success from 76.4% to 92.6% and reduce downstream API spending by an average of 19.7%. The cross‑model success‑rate spread narrows from 21.6 to 3.9 percentage points, with all four models achieving mean success rates above 90%. An ablation study reveals that removing any component degrades performance, with the largest losses stemming from the absence of code and environment.\ \ These findings suggest that preserving execution knowledge helps even models with lower raw‑code success reproduce experiments more reliably and at lower cost.\ \ Review: ReproBridge’s systematic artifact structure and auditing mechanism markedly improve reproducibility of ML experiments while cutting compute costs, offering a scalable blueprint for the research community.

Original Source: https://arxiv.org/abs/2611.08452

[h] Back to Home