Large language models (LLMs) are increasingly employed to support organizational decisions, yet users often lack a principled basis for judging whether to rely on a specific recommendation. Existing approaches typically assess broad model properties—such as reliability, uncertainty, or robustness—or focus on user trust, without clarifying the fundamental basis for depending on an individual recommendation. Drawing on epistemological theory, we introduce epistemic warrant as a decision‑level construct that captures the stability of a model’s preference and the scope over which that preference holds. We operationalize this construct via a four‑tier reliance certificate for pairwise recommendations, distinguishing unstable, context‑dependent, locally supported, and broadly supported recommendation types. Known‑groups tests validate the construct: they recover expert‑prespecified warrant orderings, and higher‑level warrants align systematically with independent consensus from crowd workers. Moreover, experiments show that epistemic warrant conveys information distinct from verbalized confidence and is not readily explained by decision difficulty. Overall, the framework offers a theoretically grounded, implementable method for characterizing the warrant of individual LLM recommendations when objective ground truth is unavailable.
Review