Faithful chart generation in real-world data‑science workflows requires grounding visualizations in scattered evidence, computing chart‑ready quantities, and rendering them accurately. Modern large language models can produce visually plausible, instruction‑compliant charts, yet data‑level hallucinations remain hard to detect in long, noisy, multimodal contexts. To quantify this gap, we introduce DEEPCHART, an expert‑annotated benchmark containing 1,482 task‑conditioned chart‑generation instances drawn from real scientific papers, financial filings, and ecosystem reports. DEEPCHART formulates chart generation as an Extract‑Reason‑Visualize pipeline, evaluating source‑data extraction, derived‑data reasoning, and chart rendering stage by stage. Experiments with state‑of‑the‑art models show that visually plausible charts often conceal data‑level errors, especially in lengthy and multimodal inputs where extraction and reasoning mistakes are frequent. Simply enlarging context windows is insufficient; faithful chart generation also demands reliable evidence extraction and quantitative reasoning before rendering. The code and data are released at https://github.com/tangdouer1005/DeepChart.
Blogger's Review: This benchmark offers a crucial yardstick for measuring LLM reliability in practical data‑visualization tasks and highlights that scaling models alone will not solve hallucination issues without strengthening evidence extraction and reasoning capabilities.