Abstract
LLM agent benchmarks typically measure task completion, reliability, and inference cost, but overlook the persistent data left on disk by an agent run, including logs, context snapshots, checkpoints, and debug traces. We introduce AgentFootprint, a cross-framework benchmark for post-run agent storage footprint.
Its serialization-aware metric suite measures total retention, channel composition, duplication, growth, compressibility, and conversation-history reconstructability. It addresses a measurement trap: naive byte-level measurement understates duplication due to database paging and JSON escaping obscuring repeated content. A fixed-trace control separates agent-generated logical volume from persistence-layer amplification: replaying the same trajectory through seven persisting frameworks yields a 6.7x spread. Under identical models, tools, and tasks, configurations with 100% accuracy differ by 15.7x in retained bytes, despite their defaults supporting different recovery and audit capabilities. Three full-history configurations grow superlinearly on a repeated-observation stress task.
Exported trajectories from 108 instance-normalized SWE-bench Verified submissions span three orders of magnitude per instance, with no detectable correlation with resolve rate. A content-addressed store reduces retention by 4.8x-32.7x while preserving every reconstructability score. These results establish persistent storage as a resource metric to be reported jointly with accuracy and reconstructability.
Blogger's Review: The AgentFootprint benchmark introduced in this paper offers a fresh perspective on storage management for LLM agents, highlighting the significance of persistent storage in performance evaluation. By quantifying storage footprints, developers can better understand and optimize resource usage, paving the way for more efficient model deployment and operation.