Large language models can produce harmful text, creating a strong demand for test‑time detoxification. Existing representation‑steering approaches often rely on static or coarse interventions and fail to model the transition from toxic to non‑toxic generations, resulting in imprecise guidance.
We introduce Transition-Aware Reward-Guided Representation Sampling (TARGRS), a framework that learns reward guidance over toxicity transitions in the latent representation space. During exploration the method steers progressively along the toxicity direction, building trajectories from non‑toxic to toxic states. Toxicity scores computed along these trajectories turn sparse annotations into dense listwise supervision for a transition‑aware token‑level reward model.
At inference time the reward model guides coarse‑to‑fine sampling in the opposite, non‑toxic direction. Coarse sampling estimates the amount of intervention required to reach a non‑toxic region, while refinement selects a detoxified representation with minimal perturbation.
Evaluations on RealToxicityPrompts, BOLD and AttaQ across ten LLMs show that TARGRS consistently outperforms state‑of‑the‑art baselines, achieving up to a 69.38% reduction in toxicity on RealToxicityPrompts. The method runs almost as fast as the original model and preserves core capabilities with negligible impact. Extending the approach to large vision‑language models also yields substantial gains, lowering toxicity by up to 64.23% on three representative LVLMs.
By intervening directly in the decoder’s latent space, TARGRS provides an efficient, low‑intrusion detoxification solution while maintaining generation quality.
Review: The approach demonstrates that transition‑aware reward signals can dramatically improve safety without sacrificing performance, offering a practical path for secure deployment of large generative models.