Diffusion language models (DLMs) generate text via bidirectional denoising and iterative refinement, yet control is still governed by token‑level masking schedules. Consequently, draft‑level refinement remains implicit, limiting explicit decisions about when to insert missing content, delete redundant spans, revisit uncertain regions, or stop editing.
We introduce the Diffusion Editing Language Adapter (DELA), a plug‑in framework that enables controllable refinement of clean, variable‑length drafts while keeping the DLM frozen. DELA defines four explicit editing actions: Insert, Delete, Remask, and Stop. A controller decides when and where to edit, and the frozen DLM fills masked regions through denoising, separating high‑level edit control from token‑level generation.
A naïve single‑phase editor would be computationally expensive because arbitrary interior edits break prefix stability and hinder KV‑cache reuse. DELA therefore adopts a two‑phase decoding procedure:
- Fast append‑only drafting phase – edits only at the end, allowing full KV‑cache reuse;
- Global refinement phase – performs targeted non‑monotonic edits on the whole draft, enabling global optimization.
The controller is trained with a staged curriculum:
- First, it learns local edit priors from synthetic recovery tasks and mined actions that improve draft quality;
- Subsequent stages refine edit selection via rollout‑based outcome preferences and alignment with downstream task feedback.
Experiments on mathematical reasoning and code generation show that, with the backbone frozen, DELA achieves up to a 5.5‑point accuracy gain over direct diffusion decoding.
Review: DELA lifts edit control to the draft level, offering a modular and low‑cost solution for controllable generation, and broadening the practical applicability of diffusion language models.