Abstract
Interactive 3D segmentation aims to extract object masks from point clouds with minimal user clicks. Despite recent advancements, existing methods struggle with:
- Coarse voxel resolution blurring fine boundaries under limited clicks;
- Hard false positives caused by confusing background structures.
These issues are exacerbated by density and scale shifts across datasets (e.g., dense RGB-D reconstructions vs. sparse LiDAR scans), where fixed refinement heuristics and purely click-driven decoding generalize poorly. To tackle these challenges, we propose NegROI—a novel transformer-based interactive framework that combines click-centric multi-resolution refinement with scene-conditioned negative prompts.
Given a coarse voxel prediction, NegROI refines only a local Region Of Interest (ROI) around the current click on a finer grid and fuses the refined logits back to the coarse mask. To enhance robustness and efficiency, we introduce uncertainty-driven selective refinement prioritizing ambiguous regions. Additionally, we model hard background patterns using a set of scene-conditioned negative prompts obtained through cross-attention over scene tokens, stabilized by a diversity regularizer. Finally, we propose boundary-aware hard negative mining to guide negative-prompt attention towards boundary-proximal, high-confidence false positives.
Our experiments on benchmark datasets (i.e., ScanNet, S3DIS, and KITTI) demonstrate improved click efficiency and reduced false positives, showcasing stronger cross-dataset robustness compared to state-of-the-art baselines.
Blogger's Review: NegROI integrates various innovative techniques, particularly in handling uncertainty and background noise, showcasing significant potential in interactive 3D segmentation. Its effectiveness is not only evident in experimental results but also provides new insights for future research, especially in the application of diversity regularization and negative sample mining, which deserves further exploration.