Scientific papers require models to jointly reason over text, equations, figures, tables, code, and datasets while preserving evidence provenance. Existing benchmarks usually test these abilities in isolation, making it unclear whether multimodal models can support realistic scientific‑reading workflows.
We propose SciDocBench, a workflow‑centered benchmark for scientific document understanding. It contains 124 expert‑authored, difficulty‑screened questions, organized into seven research‑assistant capability groups and 19 subtasks across five scientific domains. Each question is instantiated under four matched conditions—English or Chinese, and all‑images‑first or interleaved document representations—resulting in 496 evaluation instances for controlled analysis.
The strongest evaluated system scores only 62.6/100, showing notable weaknesses in document perception, evidence grounding, verification, and cross‑document reasoning.
To turn these diagnostics into scalable training signals, we introduce SciDocIR, a typed evidence‑graph representation that preserves scientific document objects, layout and cross‑reference relations, and provenance.
Based on SciDocIR we build SciDocDataset, comprising roughly 15 K supervised fine‑tuning samples and 8 K reinforcement‑learning samples covering 14 verifiable subtasks.
Together, SciDocBench, SciDocIR, and SciDocDataset constitute an evaluation‑to‑training framework for diagnosing and improving scientific‑document assistants. The project page is https://github.com/InternLM/SciDocBench.
点评