In rectified-flow-based generative models, neural networks are trained to predict two different targets, such as instantaneous velocity or data endpoints, for denoising. Although prior work shows that these parameterizations lead to different empirical behaviors, the underlying mechanisms of their advantages remain underexplored, and effective combination methods are unclear.
This work analyzes how learning errors from different parameterizations affect generation performance. We show that predicting data endpoints provides a clear training signal that stabilizes training, while predicting velocity maintains stable sampling dynamics near the data manifold.
Motivated by these insights, we propose Self-Consistent Flow (SC-Flow), a new method that unifies the benefits of both parameterizations. By employing a lightweight consistency loss, SC-Flow jointly trains a single network to predict both local velocity and data endpoints, with consistency between the two predictions enhancing model performance.
The method requires no major architectural changes and adds minimal computational overhead. Extensive experiments on image generation tasks demonstrate that SC-Flow substantially stabilizes optimization and improves the straightness of generation paths, leading to significant gains in generation quality over standard rectified-flow baselines.
Blogger's Review: The SC-Flow method showcases innovation in generative models by jointly training local velocity and data endpoints. Its lightweight consistency loss design effectively enhances model performance, making it a promising approach for further exploration and application in various fields.