Diffusion language models can generate tokens in parallel, yet they still need multiple sampling steps to reach high quality. This work asks whether a model can produce an entire sequence in a single forward pass and introduces the Seq-Drifting framework. Seq-Drifting applies drifting to continuous token representations, learning a direct map from Gaussian noise to text. Unlike conventional DLMs that supervise each position with the observed token, Seq-Drifting builds a drift field whose position-wise targets adapt to the current generated context. At each position, attraction pulls the output toward context‑compatible token embeddings, while repulsion prevents sample collapse and intra‑sequence repetition. The generator learns this drift field during training and, at inference, jointly emits all tokens in one pass. Empirically, Seq-Drifting achieves a one‑step perplexity of $36.01$ on LM1B and remains competitive in both unconditional and conditional generation. Additional experiments on translation, summarization, and reasoning confirm the framework’s broader applicability. The code is released at https://github.com/jyliu-ai/Seq-Drifting.
Review