Large language models (LLMs) can expose memorized training sequences through prefix‑based extraction: given a prefix from a training example, the model may assign high probability to the original continuation. In real‑world deployments, prefixes are rarely evaluated in isolation; they usually appear together with instructions, retrieved documents, or other task‑specific context, as in retrieval‑augmented generation (RAG). This raises a key question: does contextual conditioning mitigate memorization, or does it merely shift the set of memorized samples that become extractable?
We conduct paired item‑level measurements of probabilistic suffix extraction. For each prefix‑suffix pair, we score the target suffix under an empty prompt and under retrieved contexts of varying relevance, using three open‑weight instruction‑tuned models. The results show that context does not simply erase memorization. Extractable memorization consists of a context‑robust core and a context‑sensitive boundary. As prefix length increases, many samples that are extractable without context remain extractable under retrieved context. At the same time, context mainly affects marginal samples near the extraction threshold: it suppresses some exposures while enabling new ones missed by prefix‑only evaluation.
These findings qualify the view that RAG reduces memorization risk. Context can lower aggregate extraction by suppressing boundary cases, yet robustly extractable samples persist, and context‑enabled extractability remains security‑relevant.
Review