Abstract
High-temperature sampling is one of the primary mechanisms for increasing diversity in LLMs. Recent advances in truncation-based sampling techniques have helped mitigate drawbacks of high-temperature sampling, such as neural text degeneration, thereby enabling greater diversity in LLM outputs without sacrificing coherence. However, increasing the entropy of the token probability distribution via high temperatures has been shown to weaken model guardrails by reducing the model's refusal response in the presence of harmful prompts. Despite the potential benefits of high-temperature sampling and the importance of maintaining model safety, there is a lack of existing solutions for maintaining the refusal behavior of LLMs under a higher entropy regime.
To address this gap, we systematically study how temperature influences refusal behavior in LLMs and propose an efficient sequential decoding approach that preserves a model's greedy decoding refusal response at high temperatures while incurring minimal additional latency. Through extensive experiments, we show that our approach preserves 91-99% of the greedy decoding refusal behavior across three benchmark datasets without compromising the model's high-temperature response for safe prompts. Our work demonstrates how refusal behavior can be maintained efficiently for applications requiring high-temperature sampling.
Blogger's Review: This research provides crucial insights into model safety under high-temperature sampling. By introducing the refusal-gated decoding method, it significantly enhances LLM's protective capability against harmful prompts. The method's effectiveness and low-latency characteristics lay a foundation for future safe applications, showcasing the potential balance between diversity and safety.