Abstract
Modeling psychological disorders in artificial agents offers both a testbed for computational psychiatry and a lens on the failure modes of affective control. Prior work induces one or two disorders in a reinforcement learning (RL) agent by hand-tuned reward shaping, labels the behavior post hoc, and reports single runs. We recast disorder modeling as dose-controllable manipulation of cognitive appraisal signals in an appraisal-guided PPO agent, expressing seven disorders (anxiety, mania, obsessive-compulsive checking, depression, impulsivity, addiction, and post-traumatic stress) each as a single knob grounded in a computational psychiatry account, with each symptom measured by a preregistered assay mapped to a recognized paradigm. Across more than a thousand runs (10 seeds, four controls, 95% confidence intervals), every disorder shows a graded, monotone dose-response that no control reproduces.
Beyond these induced effects, three findings emerge that were not written into the reward:
- The disorders self-organize into a two-dimensional affective space in which mania mirrors anxiety;
- Removing a knob remits reward distortion disorders (mania, checking, addiction) but not avoidance disorders (anxiety, PTSD), which instead recover under a graded exposure curriculum;
- Two simultaneous knobs interact nonadditively, yielding testable comorbidity predictions.
Appraisal weights thus parameterize a controllable space of affective phenotypes in which the same knobs that induce a disorder can model its treatment. We also show that three disorder knobs (depression, addiction, anxiety) transfer to a three-dimensional pixel environment (MiniWorld) with a standard convolutional agent and no appraisal critic, with cross-assay dissociation confirmed across both domains, indicating the framework is not specific to grid worlds or to PPO's appraisal critic.
Blogger's Review: This study successfully simulates multiple psychological disorders in reinforcement learning agents by introducing controllable cognitive appraisal signals, providing new perspectives for computational psychiatry. Its findings not only aid in understanding the interrelations of disorders but also open up new possibilities for future therapeutic approaches. Notably, the research highlights reproducibility and cross-domain effectiveness, emphasizing the broad applicability of the framework.