Abstract
Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025) dataset represents a significant step forward, yet its coverage is still limited to 18 high-resource languages.
To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional underrepresented languages spanning 6 language families, ranging from mid-resource to extreme low-resource settings. We constructed the dataset through a human-curated pipeline, where native speakers thoroughly validated pre-generated translations.
Using PluraMath, we benchmarked 27 reasoning LLMs across four model scales — small, mid-size, large, and closed-source ensembles — probing the multilingual mathematical reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical reasoning performance between high-resource and underrepresented languages, with stronger results largely associated with better instruction-following ability.
We fully open-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities.
Blogger's Review: The launch of PluraMath is a significant step towards supporting mathematical reasoning evaluation for low-resource languages, promoting fairness and diversity in language models. This initiative will encourage more researchers to focus on neglected languages, contributing to the healthy development of the overall AI ecosystem.