Video diffusion models often violate fundamental physics when generating fluid content, such as liquid columns breaking apart in mid-air and water levels in containers not rising. This gap arises from the lack of explicit motion supervision in large-scale video-text corpora, leading models to learn fluid appearance rather than dynamics. We address this with two key contributions.
First, we constructed a physics-simulation fluid dataset, combining 1,638 MPM-simulated pouring/sloshing videos with 2,320 keyword-filtered real pouring videos, alongside two held-out test sets: a 1,515-video real-video benchmark and an 18-prompt text-to-first-frame generalization benchmark.
Second, we introduced a dual-stream image-to-video architecture based on a pretrained diffusion-transformer video generator. This architecture augments the standard RGB decoder with a lightweight Optical-Flow Decoder branch trained with explicit end-point-error and smoothness losses, fused into the RGB stream via zero-initialized convolutions to keep the pretrained backbone undisturbed. Only the two decoders are updated while the encoder, temporal transformer, and text encoder remain frozen.
Across two model scales (1.3B and 14B) and two test sets, our method improves VideoPhy-2 Physical-Commonsense and Video-Quality scores over the frozen backbone by up to 8.75 and 4.65 points, outperforms a leading open competitor, and is preferred by human raters in a blind study. A direct optical-flow read-out evaluation further shows an end-point error as low as 0.54 pixels in-distribution, confirming that the model has internalized a coherent motion prior rather than merely improving surface appearance.
Blogger's Review: This paper significantly enhances the physical consistency of fluid video generation by integrating a physics simulation dataset with an innovative dual-stream optical flow architecture. It showcases the potential of deep learning in simulating complex dynamics, providing a crucial technical foundation for future visual generation tasks.