Scientific research involves complex information-seeking and reasoning workflows across heterogeneous sources. However, existing benchmarks primarily emphasize general-domain retrieval or static scientific question answering, failing to assess key capabilities required in realistic scientific research workflows. We introduce SciExplore, a benchmark designed to evaluate the scientific information-seeking and reasoning capabilities of LLMs and agents.
SciExplore comprises four task types covering 103 expert-curated tasks across more than ten scientific disciplines: scientific database navigation, ambiguous literature retrieval, missing reference completion, and cross-source structured knowledge synthesis. These tasks probe progressively higher-level abilities from entity-level reasoning and document-level identification to evidence-level grounding and domain-level synthesis.
We evaluate over ten state-of-the-art LLMs and autonomous agents on SciExplore, revealing substantial performance gaps, with performance degrading sharply as task complexity increases and extremely low accuracy on the most challenging structured synthesis tasks. These results highlight significant limitations of current models and agents in realistic scientific information-seeking scenarios.
Blogger's Review: The introduction of SciExplore provides a new evaluation framework for the field of scientific information retrieval, effectively revealing the shortcomings of current models. As the complexity of scientific research continues to rise, future agents will need stronger reasoning and integration capabilities to meet the evolving demands of research.