Long‑conversation memory benchmarks increasingly test recall and prompted knowledge updates, and recent work studies evolving user beliefs and memory state. TWIST proposes to measure a complementary, previously unmeasured property—intervention quality, i.e., whether a deployed memory system intervenes correctly at belief‑change points.
Four tracks cover unprompted tension detection, vetting outgoing drafts against the record, answering with current beliefs while preserving supersession history, and governing sensitive recall. The suite extends LoCoMo’s corpora and harness; each detect/block metric is paired with a do‑not‑over‑detect control: surface‑matched hard negatives penalize false interventions so no track can be gamed by flagging everything.
The benchmark itself is validated through independent gold‑blind double annotation, adjudication, judge‑decoy calibration, and a separability audit. On the human‑validated Track B v1.0 key (161 items, post‑adjudication κ = 0.85) no configuration simultaneously achieves high contradiction recall, high hard‑negative specificity, and high attribution.
Flat‑RAG baselines detect 0.76‑0.97 of true contradictions but falsely flag 16‑43% of surface‑matched safe drafts depending on backend, while a deployed coherence‑oriented system almost never over‑flags (specificity 0.98‑1.00) yet catches only 42% of true contradictions—a trade‑off invisible to recall‑only scores.
A 13‑configuration baseline ladder localises causes: every gold contradiction is detectable from its evidence alone (recall = 1.000), calibrated models nearly solve the track given the full transcript—consistent with substantial retrieval‑coverage gaps—and draft‑only floors reveal model‑dependent style priors.
A system’s TWIST profile, beside its recall score, measures whether memory knows when to intervene and when not to.
Review: TWIST adds a fine‑grained dimension to evaluating conversational memory systems, highlighting the balance between the cost of false interventions and the benefit of correct belief updates.