Abstract
Digital mental health interventions (DMHIs) offer scalable support, but ensuring they accurately detect users' intent during volatile situations can be challenging. Pure parametric Large Language models (LLMs) lack specific safety-critical architecture, which can lead to missing critical cues or hallucinations, undermining reliability. Retrieval Augmented Generation (RAG), which supplements an LLM with retrieved context, could enhance intent detection during volatile situations.
Commercially available DMHIs typically combine multiple independent safety layers like rule-based filters, symbolic escalation protocols, and neural classification. However, the incremental contribution of any single layer remains unquantified. This paper evaluates six LLM models within a DMHI called Wysa, via a controlled comparison of RAG-enabled versus RAG-disabled modes. Anonymized real and synthetic user-chatbot exchanges were annotated by a qualified clinical team against multi-class intent categories (e.g. self-harm, abuse, panic). The study computed classification accuracy, recall, precision, and F1 scores against ground truth labels and tested differences for statistical significance. Performance was also examined by risk category and inter-model agreement.
While RAG caused a rise in false alarms, the trade-off is consistent with safety-critical design principles that prioritize sensitivity, where flagged cases are routed to additional review rather than acted on directly. Overall, these findings support RAG as a promising approach to improve the accuracy, consistency, and safety of LLM-driven DMHIs.
Blogger's Review: This empirical study supports the application of RAG in the mental health domain, highlighting the importance of sensitivity in safety-critical design. The findings provide significant insights for future digital mental health interventions and warrant further exploration and optimization.