Existing hallucination detection methods are typically conducted at the inference stage without modifying the model itself. This paper explores fine-tuning strategies that enhance the detectability of hallucinations in the model.
Focusing on semantic-entropy-based detection, we observe that many erroneous outputs remain undetected because the model produces nearly identical incorrect answers across multiple runs. To address this, we propose diversity-oriented fine-tuning to encourage more varied generations.
We introduce two specific strategies: one based on Supervised Fine-Tuning (SFT) and the other on Direct Preference Optimization (DPO). Extensive experiments are conducted to evaluate our approach and analyze the behavior of the models before and after fine-tuning.
We find that after adopting our fine-tuning methods, the models become less likely to produce low semantic entropy responses for hallucinated answers, thereby improving the effectiveness of hallucination detection, eventually yielding results better than or comparable with state-of-the-art methods. The code will be publicly released.
Blogger's Review: The diversity-oriented fine-tuning strategy proposed in this paper offers an innovative approach to hallucination detection by encouraging models to generate more varied outputs, significantly enhancing detectability. This work holds substantial practical value and potential for future applications. Looking forward to the open-source release of the code.