This paper presents a complete technical solution for the KuaiRP series of role‑playing models, targeting four core objectives: simplified prompt engineering, highly stable output quality, built‑in domain knowledge, and high‑efficiency deployment with a small parameter count. Injecting deep domain knowledge typically causes catastrophic forgetting of the model's general agent abilities, so we devise a multi‑stage training pipeline.
The first stage defines a standardized character template and builds an SFT data pipeline using user‑behavior simulation and reverse profile filtering, ensuring high‑quality alignment of role information.
In the second stage, a rule‑based composite reward function is applied during reinforcement learning to curb common degradation such as length expansion and repetitive generation, preserving coherence and diversity of the generated text.
The third stage introduces a novel self‑distillation paradigm: Two‑stage On‑Policy Distillation (OPD) equipped with Cumulative‑Divergence Decay (CDD). The domain‑adapted model serves as the teacher and the original base model as the student; OPD‑CDD simultaneously restores the general capabilities weakened during SFT and RL, achieving a balance between deep domain knowledge injection and general ability preservation.
Experimental results demonstrate that KuaiRP models match the state‑of‑the‑art proprietary models in role‑playing fidelity within target domains, successfully recover general agent capabilities, and maintain extremely low deployment costs.
Review