Large Language Models (LLMs) achieve impressive results on tasks such as summarization and question answering, yet their internal decision‑making remains opaque. Mechanistic interpretability (MI) seeks to reverse‑engineer neural networks into algorithms that humans can understand. Most MI pipelines for LLMs follow a two‑stage process: first identify important components (circuit discovery), typically individual attention heads or feed‑forward neurons; then interpret the functional role of those components for a given task. This sequential approach neglects the fact that a component's importance and its functional role are mutually dependent. Unifying the stages raises two challenges: (1) functional roles are often tied to specific nodes, limiting cross‑task generalization; (2) role identification relies on subjective interpretation rather than quantitative metrics. To overcome these issues we introduce S^3martCirc (Self‑supervised Smart Circuit Discovery), a unified framework that discovers circuits and interprets functionality simultaneously. S^3martCirc abstracts node behavior into two generic computational roles that generalize across tasks and defines a quantitative metric for role assignment, allowing importance and function to be learned jointly. Extensive experiments demonstrate that the proposed method outperforms existing approaches in circuit discovery.
Review