Continuous diffusion models have emerged as a promising non‑autoregressive paradigm for text generation over continuous embedding spaces. Along the continuous trajectory, the residual token identity directly shapes the denoising objective, yet the geometric trajectory misaligns with temporal progression, causing severe temporal compression. We show that embedding norm decay makes Euclidean distance produce spurious overtaking, indicating that token competition must be evaluated directionally.
In this space, a candidate overtakes the source token only when the effective noise magnitude exceeds their semantic separation. We formalize these pairwise dynamics as Competition Threshold Theory (CTT). High‑dimensional overtaking requires extreme noise levels, causing standard trajectories to concentrate identity decay into a narrow window near the noisy endpoint.
To address this, we propose Semantic Retention Scheduling (SRS), aligning time allocation with the progression of identity decay, and demonstrate that existing token identity metrics are fundamentally statistical aggregations of these pairwise crossings. Building on these insights, we introduce the LLaCO framework. LLaCO adapts a pretrained backbone to continuous representations via an embedding directional adapter and timestep‑conditioned modulation. To operate within the directional space, LLaCO employs a spherical noising process formulated as a geodesic interpolation toward a mask‑biased endpoint, eliminating norm decay while preserving accurate token competition.
Guided by SRS, LLaCO reallocates the denoising trajectory proportionally to the logarithmic rank decay. Instantiated as an 8B model, LLaCO outperforms previous continuous diffusion baselines and rivals discrete diffusion models of comparable scale.
Review