NeFut Logo NeFut
中 Admin Login

[CS.AI] Testing, not presuming, adequacy: calibrating generative social simulators against emergent network structure

Published at: 2026-09-24 22:00 Last updated: 2026-09-28 00:49
#algorithm #AI #Machine Learning

Validation of generative social simulators usually stops at face validity, merely describing emergent network structure without quantifying parameter uncertainty or checking adequacy.

We therefore propose an adequacy‑aware calibration protocol that combines amortized posterior estimation, synthetic identifiability assessment, matched‑sample‑size adequacy check (prior‑predictive reachability plus per‑statistic posterior‑predictive localization), diagnosis‑guided repair and statistic‑held‑out audit.

The protocol is demonstrated on a real second‑hand luxury resale market split into four channel‑by‑residency cells, each a bipartite buyer‑brand network. The forward model is built from persona profiles elicited offline once by a language model. Behavioural parameters are recoverable in all cells, though calibration is approximate and overconfident for one parameter.

The observed network summary lies outside the simulator’s reachability reference in every cell, with the mean purchased tier being the dominant discrepancy. The repair satisfies the value‑block criterion in two cells but does not restore overall adequacy; the held‑out audit uncovers a buyer‑breadth dispersion miss that earlier diagnostics missed.

An ablation of profile sources shows that language‑model profiles outperform a flat rule baseline in all cells, while relabelling brands within categories causes no consistent degradation, indicating that the profiles’ value resides in structural information rather than brand identity.

In sum, without making causal claims we conclude that an independent‑aggregation account lacking agent interaction or a buyer‑breadth mechanism cannot jointly reproduce the market’s purchased‑tier level, head‑brand concentration, community structure and buyer‑breadth heterogeneity.

Review: The study offers a systematic adequacy‑aware calibration framework, highlights both its feasibility and its limits on a real social network, and provides a useful reference for future evaluation of generative simulators.

Original Source: https://arxiv.org/abs/2609.24012

[h] Back to Home