Diffusion models (DMs) have achieved remarkable performance in high‑fidelity, controllable generation, following the pretraining‑finetuning paradigm popularized by large language models (LLMs). Despite these advances, conventional DMs struggle with sophisticated reasoning, autonomous exploration of generation pathways, and alignment with complex human‑defined objectives. Inspired by the success of RLHF in LLMs, reinforcement learning (RL) has emerged as a post‑training mechanism that iteratively refines DMs via reward‑driven trajectory optimization. This paper offers the first comprehensive review of RL‑driven DMs and empirically examines how RL reshapes the diffusion process toward experience‑centric, goal‑directed generation. We present a technical panorama that clarifies RL’s multifaceted roles in language, multimodal, robotic, and scientific models, with particular focus on diffusion trajectory optimization and reward engineering. Extensive experiments on diverse benchmarks evaluate representative methods, intuitively highlighting the benefits of various RL post‑training strategies. Moreover, we identify three critical bottlenecks hindering real‑world deployment of RL‑DMs: fragility of reward design, exploration‑exploitation trade‑offs in high‑dimensional generation spaces, and training instability under sparse rewards. Building on these insights, we propose a principled roadmap for integrating RL into DMs and outline promising research directions for robust, interpretable RL‑diffusion systems. A public repository continuously tracks the latest developments: https://github.com/MAIR-Lab-HUST/Awesome-RL-for-Diffusion-Models.
Review