Most safety guardrails for large language models (LLMs) evaluate each prompt-response pair in isolation, missing failures that occur only over a dialogue. We term this Conversational Risk Accumulation (CRA), which includes gradual intent drift, fragmented assembly of prohibited instructions, and sensitivity build-up from repeated disclosures.
To address this, we propose a session-layer CRA Framework that tracks three trajectory signals:
- Semantic drift from a session anchor;
- A sensitivity-weighted information accumulation graph over extracted entities;
- A compliance-gradient signal capturing increasing willingness to comply.
For scoring, we provide (i) an unsupervised convex fusion for attribution and ablations, and (ii) CRA-Net DA, a compact learned trajectory model trained with family-adversarial objectives to reduce length and topic-coverage confounds.
To benchmark CRA, we release CRA-Bench v0.1 (1,200 eight-turn sessions across three threat families with topic-matched benign twins), CRA-Bench v0.2 (LLM-paraphrased variants to reduce template artifacts), and an extended 5-family set (2,000 sessions adding persona priming and context stuffing).
We introduce a trajectory-native evaluation protocol with session-level splits, mixed-set threshold calibration, Trajectory AUROC, turns-to-detection, calibrated false-positive metrics, bootstrap confidence intervals, leave-one-family-out diagnostic stress tests, and synthetic-to-human transfer checks. Claims focus on within-distribution session scoring on CRA-Bench and human-transfer subsets.
Blogger's Review: This study introduces a novel conversational risk accumulation framework that addresses potential safety hazards in multi-turn dialogues. By emphasizing the significance of conversational context and employing innovative scoring and benchmarking techniques, it advances the research on LLM safety. Its methodologies will play a crucial role in future LLM applications, particularly in scenarios involving sensitive information.