Large language model (LLM) agents can solve complex interactive tasks, yet reasoning and tool calls consume time. In continuously changing environments, an otherwise correct action may arrive too late. To address this, we introduce RealtimeBench, which contains 180 tasks across six game types and keeps the games running during inference to measure agents' real‑time performance.
Paired evaluations show that when games are paused, GPT-6.1‑sol outperforms the faster Jev, but when games keep running, Jev performs better. To improve real‑time capability we propose Two‑Timescale Offline Recursive Self‑Improvement (TORSI): a slow teacher studies failures, tests corrections in paused games between attempts, and writes reusable instructions; a fast student follows these instructions during gameplay without further teacher calls. Validation scores decide which updates are adopted. After two TORSI rounds, Jev's average test score rises from 13.51 to 24.19 on a 0‑100 scale, improving all six game types. The same instructions also boost the average scores of three other models without retuning.
These findings suggest that organizing deliberation, learning, and action across timescales is essential for real‑time intelligence, enabling powerful LLM agents to remain effective in a continuously changing world.
Review