Advancing language models for specialized scientific tasks requires systematic evaluation of scientific fluency, physical reasoning, and quantitative analysis. Using inertial confinement fusion (ICF) as a testbed, we built an expert‑curated benchmark of 250 questions that span qualitative reasoning, numerical calculation, symbolic derivation, and equation formulation. Model responses were scored through deterministic numerical checks, symbolic verification, and language‑model judges. We compared a range of off‑the‑shelf closed‑source and open‑source large models, focusing on Prospero—a model refined for ICF via domain‑adaptive pretraining and supervised fine‑tuning. Initial retrieval‑augmented evaluations were also performed. Prospero improved its overall score by 10.7 percentage points over its base model, with gains across all question types. Frontier models achieved the highest aggregate scores, yet critical physics errors persisted, especially in numerical calculations. Retrieval helped a smaller model but did not yield a statistically significant boost for a frontier model. Two scientific stress tests highlighted useful capabilities while underscoring the need for expert assessment of physical assumptions and model validity. Based on these findings, we recommend assessing existing domain capabilities and retrieval benefits before committing to extensive specialization.
Review