NeFut Logo NeFut
中 Admin Login

[CS.AI] CLADE: Category-Level Answering from Deficient Evidence in Retrieval-Augmented Language Models

Published at: 2026-11-12 22:00 Last updated: 2026-11-13 06:56
#AI #Machine Learning #LLM

Retrieval‑augmented systems often fetch passages that are relevant but only state the answer at a coarse granularity. For instance, a query may ask for the specific disease a drug treats (familial hyperlipidemia) while the retrieved text mentions only the broader disease family (hyperlipidemia). Prior work treats such passages as insufficient; we instead define a distinct case called resolvable insufficiency: the text alone cannot resolve the fine‑grained answer, yet the mentioned category maps to a short candidate list in a domain ontology.\n\nWhen language models read family‑only passages they exhibit a phenomenon we name Clade (Category‑Level Answering from Deficient Evidence): the model tends to guess instead of abstaining or searching further.\n\nOn biomedical questions from PrimeKG, family‑only passages reduce abstention by 13.8 points compared to uninformative passages, but the resulting guesses are wrong about half the time. A counterfactual test shows that replacing an exact disease name with its broader family lowers abstention by 17.4 points relative to deleting the sentence, confirming that the family cue drives the guessing behavior.\n\nSimple mitigations—confidence filtering, scaling to a larger model, or explicitly instructing the model to abstain—do not eliminate Clade, and the effect replicates on ecological species‑interaction data in GloBI.\n\nWe then examine whether an ontology can turn coarse passages into exact answers. Given the correct family, an ontology list contains the true answer in 98.5% of cases, yet string matching finds that family on the page for only 19.8%. Allowing the model to first select the family raises this to 30.0% and improves end‑to‑end recovery from 10.0% to 26.8%.\n\nOverall, the results indicate that when evidence only provides a category, language models should stop guessing and consult an ontology; however, the steps required for a reliable pipeline remain unsolved.\n\nReview: The paper uncovers a systematic bias of retrieval‑augmented models toward over‑confident guesses on coarse evidence and proposes ontology‑driven refinement as a promising direction, highlighting an important frontier for trustworthy question answering.

Original Source: https://arxiv.org/abs/2611.14355

[h] Back to Home