In Computer-Aided Design (CAD) floor plans, graphical primitives and textual annotations provide complementary geometric and semantic cues for intelligent design understanding. With the increasing demand for industrial digitalization and deep learning-based automation, panoptic symbol spotting has become crucial. However, most existing methods remain primarily primitive-centric, underutilizing the semantic value of textual annotations. Even the few text-aware approaches often treat annotations superficially, failing to properly model the complex syntax and hierarchical semantics of CAD annotations, resulting in semantic loss and suboptimal spotting performance.
To address these limitations, we propose TextCAD, a multimodal framework that jointly models graphical primitives and textual annotations for panoptic symbol spotting. Specifically, we design a Type-Attribute Correlation Encoder (TACE) to explicitly encode the compositional semantics within annotations by jointly modeling their types and attributes. We further introduce a Semantic Hierarchy Alignment framework with Multi-level Semantic Filtering (MSF) and primitive downsampling, which adaptively aligns annotation semantics with graphical primitives at different semantic levels, enabling accurate cross-modal semantic injection and fusion. Experiments on real-world building-design datasets show that TextCAD effectively improves symbol spotting performance and achieves state-of-the-art results.
Blogger's Review: This study significantly enhances symbol spotting capabilities in CAD floor plans by integrating graphical and textual information, showcasing the potential and application prospects of multimodal learning.