The rapid growth of academic peer review has led to the exploration of large language models (LLM) as review assistants. While LLMs can produce fluent text, they often generate unsupported statements that compromise review reliability. Existing hallucination benchmarks are not tailored to peer review because verification requires grounding claims in lengthy technical papers.
HalluPeer addresses this gap by offering aligned triples of paper content, human-written reviews, and hallucination‑injected reviews, each annotated for detection, classification, and localization. The pipeline first derives a peer‑review‑specific hallucination taxonomy, then pinpoints review contexts, and finally injects hallucinations using automated filtering.
Experiments on 12K papers and 38K reviews reveal that current detectors struggle to separate hallucinations from legitimate critique. Evaluation on authentic reviews shows that the hallucination patterns defined by HalluPeer occur in real peer reviews, underscoring the urgent need for source‑aware verification. The project repository is https://github.com/Lin-TzuLing/HalluPeer.git
Review