This paper introduces a unified stage division and a six‑dimensional analytical framework to systematically capture the entire training pipeline of large language models (LLMs) and multimodal LLMs (MLLMs). The dimensions are data, objective function, optimization strategy, system constraints, evaluation, and risk governance. We review 395 core references from 2017 to 2026, constructing a cross‑stage conceptual map: pretraining imposes forward constraints on post‑training, while post‑training can retroactively empower data collection and pretraining design. Three key findings emerge:
- The comparability of scaling laws hinges on data quality and mixture, a factor long overlooked.
- Preference alignment methods (e.g., RLHF, DPO) and reasoning reinforcement learning with verifiable rewards reshape the boundaries of post‑training, yet the Pass@1 vs. Pass@K debate remains unresolved.
- Multimodal training introduces structural challenges to the pure‑text paradigm in tokenizer design, training objectives, and evaluation metrics.
The paper also supplies a cross‑stage effects diagram, a method‑family map, a practical decision tree, and at least ten verifiable open problems, together with an extensible skeleton for focused sub‑surveys per stage.
Review