Large language model (LLM) agents often excel on isolated tasks but tend to drift into inconsistency during prolonged interaction. We systematically evaluate temporal consistency in a controlled 20‑step multi‑agent environment inspired by delayed‑gratification studies.
Experimental design: at each step the agent chooses between continuing to delay a reward or claiming it immediately (terminating the episode). Factors manipulated include social visibility (private vs public), persona stressors, and deliberation policy. A total of 84,540 trajectories were collected across eight model families.
The first reward claim is treated as a time‑to‑event outcome. We estimate Kaplan‑Meier survival curves and fit a discrete‑time hazard regression to quantify how experimental factors shift failure risk over time.
To analyze failure rationales and language patterns, we built a seven‑category taxonomy from 13,780 deliberation traces of agents that terminated the episode. Labels were generated with LLM assistance and audited by humans, achieving an inter‑rater agreement of $\\kappa = 0.83$.
Findings show that early failures are predominantly impulse‑driven, while later failures are framed by fatigue and cost‑benefit considerations; public settings increase norm‑oriented justifications. Moreover, we observe a deliberation‑inconsistency link: longer deliberation correlates with higher intra‑rationale contradiction rates, challenging the assumption that more reasoning text implies greater consistency.
Together, the survival and rationale analyses reveal distinct temporal reliability regimes and model‑specific “failure fingerprints,” offering a new evaluation lens for diagnosing inconsistency in multi‑turn agent behavior.
Review