Driven by next-token prediction, NLP has transitioned from task-specific models to powerful generalist foundation models. So, what equivalent catalyst is needed to achieve a general-purpose model in computer vision? This paper argues that large-scale text-to-video generation serves as a strong pre-training paradigm for computer vision, providing the necessary spatiotemporal priors, vision-language alignment, and scalability for general visual intelligence.
We introduce GenCeption, which leverages a pre-trained video generative diffusion backbone to define a feed-forward perception model capable of performing various vision tasks steered by text instructions. Empirical results show that GenCeption achieves state-of-the-art performance across diverse tasks, including depth, surface normal, and camera pose estimation, expression-referring segmentation, and 3D keypoint prediction, often matching or surpassing specialized models (e.g., DepthAnything3, SAM3, D4RT, VGGT-Omega, Sapiens, David, Genmo, and Lotus-2).
Moreover, the video generative pretrained backbone outperforms alternative pretraining paradigms (e.g., V-JEPA and Video MAE) under comparable settings. Importantly, GenCeption exhibits preliminary data and model scaling properties with exceptional data efficiency, achieving comparable performance with leading models like D4RT and VGGT-Omega with 7 to 500 times less training data.
Finally, GenCeption also exhibits intriguing emergent behaviors: a model trained exclusively on synthetic human videos generalizes to real-world footage and out-of-distribution object categories (e.g., animals and robots). These findings suggest that video generation is not merely a synthesis tool but a foundational path toward generalist vision intelligence for the physical world.
Blogger's Review: This research showcases the immense potential of pre-training via video generation models in the computer vision domain, indicating the possibility of bridging synthesis and reality, especially in data-scarce scenarios. In the future, video generation may become a key technology for achieving broader visual intelligence.