Memory and RAG evaluations often treat the model’s input as an implementation detail, overlooking that the same conversation history can be rendered to the reader as a memory entry, a summary, a typed record, or a raw excerpt. To address this, we introduce RENDER, a benchmark control that keeps the underlying dialogue fixed while systematically varying the reader‑facing artifact.
RENDER combines a five‑level packet ladder that pinpoints when answer‑bearing content enters the input, with deterministic templates that approximate ChatGPT‑style entries, LangChain summaries, MemGPT‑style typed records, and raw conversation. We evaluated 500 LongMemEval questions across nine models. Matched‑budget "packet" versions outperformed recency‑truncated raw dialogue by 42.4–72.6 points. Within deployment‑style templates, the best‑worst spread per model ranged 24.6–48.8 points; under the primary scorer, ChatGPT‑style entries scored higher than raw conversation on 7 of 9 models.
Judge rescoring preserved the overall positive effect, though model‑specific significance was mixed. Notably, three models that scored 0 % on formal ledger packets answered the same facts from natural‑language entries at 45.4–53.4 % accuracy. This effect persisted under retrieval noise and transferred to HotpotQA, suggesting that memory/RAG evaluations should report or control the reader‑facing artifact.
Blogger's Review: RENDER shines a light on a hidden bias in evaluation pipelines, reminding researchers that the format presented to the reader can dramatically influence measured performance. Future memory and retrieval benchmarks should explicitly specify or standardize this artifact to obtain a fair assessment of model capabilities.