Abstract
Multi-Task Learning (MTL) in robotics perception systems supports comprehensive 3D spatial scene understanding by integrating semantic segmentation and depth estimation. While Vision Foundation Models (VFMs) are increasingly adopted as robust feature encoders, existing decoding strategies present a critical bottleneck. To address this, we propose DPNeXt, a streamlined multi-scale feature fusion decoder and efficient alternative to the standard Dense Prediction Transformer (DPT).
DPNeXt utilizes dual depthwise separable inverted bottlenecks to enhance frozen VFM utilization through fusion-centric decoding and independent task modularization. To further mitigate negative inductive transfer between tasks, we introduce the Multi-Task Boundary Guidance (MTBG) strategy. Unlike prior boundary-aware methods that add fusion modules or gating, MTBG applies symmetric boundary-focused supervision to encourage geometric consistency without additional annotation or inference costs.
Experiments on Cityscapes show that DPNeXt-S outperforms prior state-of-the-art (SOTA) MTL models, while DPNeXt-B further improves overall performance, achieving the best results among the compared methods. On NYUv2, DPNeXt-B also secures the best semantic segmentation and depth estimation results among the compared methods while requiring substantially fewer trainable parameters than previous large-scale MTL models. Compared to the standard DPT, DPNeXt-S reduces trainable parameters by 78.6% and achieves the fastest inference speed among the compared models on resource-constrained laptop hardware.
The source code, model checkpoints, and a demo video will be made available at GitHub.
Blogger's Review: DPNeXt introduces the Multi-Task Boundary Guidance strategy, significantly reducing negative transfer between tasks and providing new insights for multi-task learning. Its outstanding performance in resource-constrained environments highlights the importance of lightweight model design, making it a promising focus for future applications.