Existing quality control for long‑context instruction‑tuning data relies on a single scalar judgment inside the generation pipeline, which reveals neither the document parts an answer draws from nor the number of reasoning steps it records, making the data unauditable and uncontrollable. We formalize data quality as a property of the alignment among document, question, and answer, and build LongForge. LongForge consists of two components: an analyzer that reverse‑reconstructs each sample into an evidence‑assertion reasoning graph and programmatically computes four dimensions—breadth, depth, correctness, and completeness; and a synthesizer that embeds the analyzer into generation, gating on correctness and completeness while handling breadth and depth via a training‑time mixture over measured values, thus making the training distribution controllable. Every reading can be traced back to specific evidence spans and reasoning steps, enabling full auditability. Measuring seven mainstream datasets uncovers two previously unquantified defects: breadth barely grows with document length and depth is generally low, and the two are nearly orthogonal. The synthesizer then integrates the analyzer, and on two backbones, its two mixtures (LongForge‑1 and LongForge‑N) outperform the strongest baseline by 2.2‑4.7 points on average across six long‑context benchmarks. In a controlled comparison that fixes the gate and candidate pool, raising breadth and depth alone adds another 1.16 points, demonstrating that the structural distribution of training data is a lever for downstream quality rather than merely a description of it.
Review