Large language models (LLMs) are increasingly deployed to assist consequential decisions, yet their reliability under equivalent uncertainty representations remains uncertain. Prior work has highlighted prompt sensitivity, cognitive biases, and confidence miscalibration, but it is unknown whether advice changes when the same probability is framed as a gain or a loss. To address this, we introduce FrameBench‑HPS, a controlled benchmark comprising 240 high‑stakes scenarios across healthcare, finance, education/career, and legal/consumer‑rights domains, yielding 2,880 prompts. Across seven leading LLMs, gain‑loss framing altered the recommendation category in 32.4% of complete groups, with most changes reflecting shifts in uncertainty rather than outright reversals. Human evaluation confirmed corresponding shifts in perceived risk and directiveness. Robustness tests showed that framing effects persist under conversational paraphrases and exceed ordinary repeated‑generation variability. Canonicalizing probability expressions markedly reduces framing sensitivity, identifying probability representation as a practical lever for improving consistency in high‑stakes LLM decision support.
Review