Claims about the cognitive abilities of large language models are often tested by converting behavioral paradigms into text prompts, which bypasses the visual interfaces and time constraints that human participants experience in real experiments. At the same time, AI agents have become capable of completing online behavioral tasks and generating human‑like data, raising concerns that social and behavioral science datasets may become contaminated.
We introduce CogArena, a benchmark that includes ten experimental paradigms from the social and behavioral sciences. Agents interact with the experiments through a standard web browser, using the same visual stimuli and response interfaces as online participants. The scoring procedure has three levels: first, whether the agent finished the task; second, how accurately it performed; third, whether its data exhibit the behavioral signatures that each paradigm reliably produces in people.
Across four multimodal models, each task was run ten times. The findings show that finishing an experiment and reproducing its signatures are distinct outcomes. Many sessions failed to finish, and among those that did, some reliably reproduced certain signatures while others showed none.
CogArena supplies the tasks, scoring code, and a public leaderboard for future submissions.
Review