Graphs serve as a backbone for large language model (LLM) systems, enabling structural reasoning, knowledge grounding, retrieval, and memory. In the emerging agentic era, graphs also orchestrate execution dependencies and multi‑agent communication, linking intermediate observations and retained information to subsequent decisions. To clarify graph contributions, this survey introduces an operation‑level "role‑locus" taxonomy covering reasoning, knowledge & evidence, retrieval & memory, execution, and coordination. We compare how graph observations and external computations guide decisions, how memory representations and updates support later tasks, and how dependencies constrain execution and information flow. The taxonomy distinguishes improvements in local metrics from downstream task performance while preserving the experimental conditions under which results were reported. We then integrate benchmarks, comparison designs, process diagnostics, and system costs into a unified evaluation framework. Finally, we highlight open challenges in adaptive computation and stopping, memory maintenance, and dependency‑aware recovery, offering a basis for judging when graph mechanisms are beneficial and for assessing their impact across components and stages.
Review: This survey provides a comprehensive taxonomy and evaluation framework that clarifies the multifaceted role of graphs in LLM systems and outlines key research directions.