This paper investigates whether preferences exhibited by large language models (LLMs) in standard elicitation tasks can forecast their strategic behavior in richer settings. When LLM agents interact with humans or other AIs they encounter potentially conflicting objectives such as individual welfare, collective welfare, and environmental impact. Prior work typically measures preferences in isolation using behavioral‑economic experiments or strategic games, and rarely checks consistency across contexts or predicts behavior in new strategic environments. Drawing on utility theory we propose the Preference‑Behavior Model (PBM), which augments a preference representation with a mechanism for imperfect utility maximization. Estimation is cast as an inverse‑game problem that jointly infers the preference vector and behavioral noise from observed choices. We evaluate PBM on three social dilemmas that expose competing motives: a public goods game, a collective‑risk dilemma, and a common‑pool resource game. The findings show that preferences derived from established elicitation instruments do not reliably predict behavior in these dilemmas, whereas parameters fitted in one dilemma retain predictive power in the others, indicating that PBM captures meaningful aspects of LLM decision‑making missed by standard tools. We also compare inferred preferences with persona instructions, observing overall alignment but revealing specific divergences across dimensions, models, and games. The methodology offers a framework for studying the consistency of LLM preferences, assessing alignment with intended priorities, and uncovering trade‑offs and behavioral patterns that conventional elicitation may overlook.
Review