Abstract
Document comprehension is a challenging yet impactful task for Multimodal Large Language Models, especially as these systems see growing adoption in real-world, human-centric applications. However, this adoption is limited for low-resource languages such as Bangla due to the scarcity of high-quality annotated data. To address this gap, we introduce BaFCo, a benchmark dataset for Bangla form comprehension with a focus on Document Layout Analysis (DLA) and Key Information Extraction (KIE).
BaFCo curates 200 multi-page complex Bangladeshi government forms, sourced from diverse sectors including agriculture, education, banking, and land management. To accurately capture the structural and contextual complexity of these forms, we define a fine-grained annotation schema comprising 26 types of form entities, along with a separate coarse form entity set consisting of 5 types.
We evaluate the latest MLLMs from the ChatGPT, Gemini, Claude, Qwen, and Kimi series using zero-shot and chain-of-thought prompts under both low and high reasoning setups.
Our results reveal limitations in current MLLMs' ability in comprehending Bangla forms, particularly in accurately localizing highly granular form entities. Our dataset and code are available at: BaFCo Dataset
Blogger's Review: The introduction of the BaFCo dataset is significant for research in low-resource languages, particularly in enhancing document comprehension capabilities in Bangla. This dataset provides rich annotation information and lays the groundwork for future optimizations of multimodal large language models. It is hoped that more researchers will leverage this resource to advance the processing of low-resource languages.