The 3D geometry of real-world scene data is often incomplete. Mainstream methods use depth estimators to inpaint missing structures. However, their predictions can be inconsistent with observed geometry or unreliable on out-of-distribution data. To address these issues, we propose the Neural Depth Field (NDF).
Our key insight is that a depth estimator can also serve as a scene-level implicit field. As an estimator, it adapts to the target domain by learning from observed depth data. As an implicit field, it fits the existing geometry to maintain consistency. Under this framework, NDF resolves both challenges through a single test-time optimization.
Experiments show that NDF produces high-fidelity and globally consistent geometry across diverse scene data, from indoor scans to satellite imagery. It reduces cross-view inconsistency by 63.3% and improves inpainting accuracy by 23.1%, achieving state-of-the-art performance in 3D scene geometry inpainting. The code is available at: NDF GitHub.
Blogger's Review: The introduction of NDF offers an innovative perspective on 3D scene reconstruction, significantly enhancing geometric consistency and filling accuracy by combining depth estimation with implicit neural fields. This method showcases the tremendous potential of deep learning in computer vision and paves the way for future research directions.