Abstract
Language models increasingly serve as advisory systems in maintenance operations. To prevent hallucination, recent systems ground these models in procedural documentation to constrain them to approved steps. In practice, however, operator queries frequently stray from this path, requiring models to recognize out-of-scope inputs mid-conversation, a dynamic that current benchmarks rarely prioritize.
We introduce DiagFlowBench, a dataset of 50 industrial diagnostic flowcharts from a consumer manufacturer converted into 1,676 multi-turn conversations that contrast compliant with out-of-scope utterances.
Evaluating a panel of ten commercial and open-weight models reveals high variability in abstention rates, with models commonly selecting a real but contextually inadequate step rather than fabricating facts. The inherent plausibility and authority of this mapped but wrong advice exposes a challenging vulnerability for grounding systems.
Blogger's Review: This study highlights the limitations of language models in real-world applications, particularly when dealing with off-procedure inputs. The introduction of DiagFlowBench offers a new perspective for model evaluation, emphasizing the importance of ensuring accuracy in model outputs. A detailed analysis of model performances could drive the development of more reliable diagnostic systems.