Simulating realistic patient interactions is crucial for testing clinical applications of LLMs at scale without time-consuming and costly user studies. However, existing methods often lack realism and controllability, frequently overshare information unprompted, and fail to capture the wide variability of patient behavior.
We introduce PatientsWithPersonality (PWP), a patient simulation framework that generates realistic yet diverse virtual patient responses through explicit personality parametrization over a latent patient state.
Grounded in HEXACO, a six-dimensional personality space used to quantify and parameterize human behavioral traits, our approach enables fine-grained control over conversational style, cooperativeness, and information disclosure within a unified framework.
In clinician evaluations, PWP is judged nearly as realistic as recorded human actors and clearly superior to prior simulators, while being flagged as "too informative" far less often.
Conditioning on HEXACO axes yields personas whose configured traits are recoverable by both clinicians and an autorater, spanning a substantially wider behavioral footprint than the closest baseline, and preventing oversharing.
Altogether, our framework paves the way for more accurate and informative LLM benchmarking through our realistic and steerable patient simulator.
Blogger's Review: The PWP framework's innovation in personalized patient simulation offers a fresh perspective for testing clinical applications. By utilizing HEXACO personality parameterization, it achieves higher interaction realism and control, significantly enhancing the evaluation standards for LLMs, which is of great importance for technological applications in the healthcare sector.