This work evaluates an explore‑then‑commit protocol for measurement‑efficient scientific law discovery. The protocol proceeds in three stages:
- A large language model (LLM) generates candidate hypotheses;
- A programmatic planner schedules measurement batches based on the hypotheses and collects data;
- A fresh prompt synthesizes the final governing equation from the fixed observations.
Key design elements are structured probes, automatic numerical diagnostics, restricted measurement batches, and optional interpreter access. Experiments were run on NewtonBench, comprising 576 trials across 12 physics modules, using GPT‑4.1‑mini and a medium‑difficulty GPT‑4.1 replication.
For medium‑difficulty tasks, interpreter‑enabled planners required on average 8.6 measurements per trial (GPT‑4.1‑mini) and 8.9 (GPT‑4.1), far fewer than the 22.5 and 43.0 measurements needed without the interpreter. Corresponding magnitude‑based root‑mean‑squared logarithmic error ($RMSE_{log}$) dropped from 2.514 to 0.202 (GPT‑4.1‑mini) and from 0.626 to 0.149 (GPT‑4.1).
An additional audit retained incomplete and invalid submissions for a coverage‑sensitive analysis. Symbolic‑accuracy gains were inconsistent across modules, and random acquisition performed competitively against disagreement scoring. Measurement savings were observed in every module, yet unequal batch constraints prevent attributing the savings solely to acquisition quality.
Overall, the full protocol shows promise as a measurement‑efficient configuration for noiseless direct‑equation tasks, while its causal components and generalization to noisy or broader domains remain open questions.
Review