GraphEcho is a benchmark designed to probe whether large language model (LLM) graph agents confuse repeated encounters with independent evidence. The suite keeps the evidence content constant while varying the number of graph paths and the origins of those evidences, then measures both the agents' judgments and their active exploration behavior. Controlled synthetic experiments show that judgment shifts depend on the model, yet redundant supporting paths increase the proportion of repeated walks for all frozen agents. To mitigate this, the authors introduce provenance‑aware post‑training (PAPT), which reduces revisits and improves synthetic accuracy but covers fewer distinct sources. On real scientific claims, PAPT still curtails repetition but overall accuracy drops. These results highlight a gap between efficient exploration and effective evidence use: an agent can learn to stop looping over the same source while overlooking needed information. GraphEcho thus offers a controlled way to evaluate what graph agents conclude and whether their exploration reaches diverse evidential sources.
Review