Abstract
In a multi-turn conversation, when a language model produces a response, which tokens from prior turns influenced that answer? How did those dependencies propagate across previous turns? Existing context attribution methods process the full context in a single pass, recovering surface-level dependencies but missing the layered, non-linear structure of real-world dialogues and multi-step reasoning tasks.
We introduce multi-turn context attribution (MTCA): given a target span in a model response, the task of tracing attribution backward across turns to identify not only which prior turns were directly relevant but also how those turns depended on earlier context.
We propose Tokengeist, an attribution-method-agnostic and scalable framework that recovers full dependency paths by casting attribution as a recursive traversal of a directed acyclic graph (DAG) over conversation turns.
We will release MTCABench, a benchmark of 3,845 target spans across 665 multi-turn conversations, annotated with gold provenance graphs reaching depths of up to 14, across four dependency types.
Across four open-weight models, flat attribution methods fail to recover multi-hop dependencies, achieving under 20% source recall, while Tokengeist reaches 90%. Our results reveal systematic failure modes of single-pass attribution — which we term provenance collapse — and motivate attribution methods that reason recursively across turns.
Blogger's Review: The introduction of Tokengeist offers a fresh perspective on understanding information propagation in multi-turn conversations. Its superior attribution capabilities compared to traditional methods highlight its potential for optimizing and analyzing dialogue systems, pushing forward research in the natural language processing field.