Integrating large language models (LLMs) into social simulation has generated strong enthusiasm, yet it also raises methodological and epistemological challenges. From a computational social science perspective we treat LLMs as cognitive or decision‑making components of simulated agents and argue that their behavioral and epistemic shortcomings can become systemic vulnerabilities when scaled to multi‑agent societies.
The ecosystem of LLM‑driven platforms is shifting from narrative sandboxes toward population‑scale simulations. In this context we introduce the Micro‑to‑Macro Validity Gap framework, which maps micro‑level limitations—hallucinations, stochastic inconsistency, representational biases, and alignment effects—onto agent‑level distortions such as persona convergence and further onto macro‑level risks like artificial consensus and behavioral homogenization.
Common evaluative pitfalls include the Fluency Fallacy, the mistaken belief that linguistic fluency implies correct reasoning, and automation bias, which leads to uncritical trust in model outputs.
Nevertheless, LLM‑based agents can provide operational value in specific settings such as serious games, participatory environments, and exploratory modeling, where the goal is interaction and hypothesis generation rather than confirmatory research or precise social forecasting.
Theory‑driven hybrid architectures embed LLMs within explicit, mechanistic, and reproducible agent‑based modeling (ABM) frameworks. While this improves interpretability, it does not automatically ensure epistemic control; validity must be assessed across multiple levels—environmental dynamics, individual behavior, cross‑level interactions, and aggregate outcomes. These systems also remain vulnerable to physics washing, where models default to physical intuitions at the expense of social complexity.
Review: LLMs hold promise for enriching social simulation, but researchers must guard against the amplification of micro‑level flaws, employ rigorous multi‑level validation, and combine LLMs with mechanistic models to mitigate fluency bias and systemic distortion.