In In-Context Segmentation (ICS), a model segments target regions in a query image using only a few reference images and their corresponding masks without updating any parameters. Despite recent advancements, prior ICS studies have largely overlooked a critical aspect: system robustness, which refers to the model's ability to produce stable segmentation results for the same query under different references.
In this work, we revisit ICS from a robustness perspective and introduce a novel paradigm, Concept-Guided In-Context Segmentation (CG-ICS). CG-ICS performs segmentation by extracting high-level semantic concepts from references rather than relying solely on low-level visual matching.
Specifically, CG-ICS introduces a concept reasoning module that utilizes a Multi-Modal Large Language Model (MLLM) to propose candidate concepts and a tree-search refined scoring function to select reliable textual concepts, alongside a parallel visual exemplar route that provides spatial grounding via simple context construction.
Both the textual concept and the visual exemplar are then used to activate the segmentation capability of a frozen SAM3 backbone. Extensive experiments on standard ICS benchmarks demonstrate that CG-ICS not only achieves state-of-the-art accuracy but also substantially improves robustness, resulting in a more reliable ICS system with significantly reduced variance across diverse reference choices. Code is available at GitHub.
Blogger's Review: This paper presents CG-ICS, which addresses the robustness issue in traditional context segmentation by incorporating a concept reasoning module. It showcases strong application potential in complex scenarios while enhancing segmentation accuracy, paving the way for future research directions that merit further exploration and implementation.