Large language models (LLMs) achieve strong results across many tasks, leveraging knowledge priors, in‑context reasoning, and test‑time computation. However, reasoning with a single model is prone to error propagation because later steps depend on earlier outputs, creating blind spots that are hard to reassess independently. To overcome this limitation we introduce collaborative learning, which builds heterogeneous collaborators and learns effective cross‑model collaboration. Model slicing replaces a single large model with several smaller, heterogeneous models under a comparable parameter budget, allowing prior reasoning to be re‑evaluated across different parameterizations. Pyramid supervision combines protocol distillation with step‑level supervision, continuously refining reasoning and interaction decisions. We evaluate the approach on four task categories—multi‑hop question answering, mathematical reasoning, code generation, and constrained open‑ended generation. The trained heterogeneous models match or surpass the original large model and outperform existing multi‑agent baselines (including reinforcement‑learning methods), while also delivering higher inference efficiency and exhibiting distinct collaboration patterns across tasks. These results demonstrate that learned collaboration turns model heterogeneity into complementary reasoning capabilities.
Review: Overall, collaborative learning with model slicing and pyramid supervision provides more robust reasoning, balancing performance and efficiency.