Modern large language models (LLMs) can translate natural‑language problem statements into operations‑research (OR) formulations. Existing post‑training methods such as reinforcement learning and on‑policy self‑distillation improve this ability, yet three limitations remain:
- Training relies on synthetic formulations validated by humans or stronger models, limiting scalable supervision.
- Credit assignment is either coarse—outcome rewards score an entire trajectory without pinpointing the responsible modeling decision—or expensive, requiring an extra evaluator.
- Privileged self‑distillation uses solver context unavailable at deployment, causing style mismatch.
We observe that a model can learn from solver‑artifact feedback generated by its own rollouts, making self‑distillation a practical, evaluator‑free source of dense supervision. Accordingly, we propose SOLID (Solver‑Informed On‑Policy LearnIng through Self‑Distillation). The framework proceeds as follows:
- Execute candidate programs from multiple rollouts and collect solver‑generated objectives and constraints;
- Cluster these objectives and select the majority‑group artifact as a pseudo‑reference;
- Compute group‑relative advantages for each rollout and use them to create dense self‑supervision signals for model updates.
Across several OR benchmarks, SOLID improves solution accuracy over outcome‑only group‑relative training for both general‑purpose and OR‑tuned models. The results demonstrate that solver artifacts can support scalable self‑improvement without trusted answers or external evaluators.
Review