AI interviewers are increasingly deployed in market research, public polling, preference elicitation, and social science studies, yet evaluating them is difficult because they must adapt over extended multi‑turn conversations. Traditional offline metrics fail to capture this dynamic behavior.
InterviewPlayground addresses the gap by offering a simulation environment where study participants are modeled on social‑theoretic principles. The system runs simulated interactions, then produces an InterviewReportCard that scores the AI interviewers using a suite of validated measures.
To test external validity, we conducted 15 real‑world qualitative studies involving five AI interviewers, three interview topics, and 450 human participants. Comparing the 12 quantitative metrics from human studies with those from the simulation yielded an average Pearson correlation of 0.86, indicating that the simulated evaluation reliably predicts human performance. Moreover, the simulated dialogues reproduced key behavioral findings observed in the human experiments.
These results confirm that InterviewPlayground is a valid tool for assessing AI interviewer performance and highlight a roadmap for future simulation‑based evaluation of conversational AI systems.
Review