BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

2026-07-08 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

BaFCo is a newly introduced benchmark dataset designed to improve document understanding for low-resource languages, specifically Bangla, by focusing on complex government forms. It comprises 200 multi-page Bangladeshi government forms from diverse sectors, featuring a fine-grained annotation schema with 26 entity types and a coarse set of 5 types for Document Layout Analysis (DLA) and Key Information Extraction (KIE). The dataset includes 16,382 DLA entities and 1,926 KIE key-value pairs. Evaluations of flagship Multimodal Large Language Models (MLLMs) like Gemini 3 Pro, GPT-5.2, and Claude Opus 4.6 reveal limitations in their ability to accurately localize granular Bangla form entities. While Gemini 3 Pro generally performs best in DLA and Bangla KIE, GPT-5.2 excels in English KIE. The dataset and code are publicly available.

Key takeaway

For AI Scientists and ML Engineers developing document understanding solutions for Bangla forms, you should recognize that current flagship MLLMs struggle with fine-grained Document Layout Analysis. Prioritize using coarse entity sets for DLA to improve localization reliability. While Key Information Extraction performance is generally stronger, be mindful of language-specific model variations and the potential for numerical hallucinations or subtle text errors. You may need to explore lightweight post-training or agentic AI approaches to overcome these limitations.

Key insights

MLLMs struggle with fine-grained layout analysis in low-resource languages but perform better on key information extraction.

Principles

• Coarse entity sets significantly improve DLA performance over granular sets.
• Increased reasoning effort does not consistently improve DLA geometric precision.
• Language impact is minimal for DLA but more pronounced for KIE tasks.

Method

BaFCo curates 200 multi-page Bangladeshi government forms, defining 26 fine-grained and 5 coarse entity types. MLLMs are evaluated using zero-shot and CoT prompts under low/high reasoning for DLA and KIE.

In practice

• Prioritize coarse entity sets for DLA to improve MLLM localization reliability.
• Expect MLLMs to perform better on KIE than DLA for document processing.
• Consider language-specific model tuning for KIE tasks.

Topics

• BaFCo Dataset
• Document Understanding
• Multimodal LLMs
• Bangla NLP
• Document Layout Analysis
• Key Information Extraction

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• How to Accurately Extract Structured Data from Complex Documents Using AI — Combining document parsing and structured extraction into a single LLM call improves accuracy for complex layouts.
• Bounding Box Label Propagation for Re-Annotation of Document Layout Analysis Datasets — BBLP uses multi-modal embeddings and label propagation to efficiently re-annotate object detection bounding boxes, significantly reducing manual effort.
• KazakhOCR: A Synthetic Benchmark for Evaluating Multimodal Models in Low-Resource Kazakh Script OCR — Current MLLMs struggle with OCR and language identification for low-resource Kazakh scripts, especially Arabic, performing worse than traditional OCR.
• VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents — Image editing models require specialized benchmarks to accurately assess their performance on multilingual and densely textual visual documents.
• Technical advances in document understanding — Document processing has evolved from basic OCR to sophisticated multimodal models that preserve structure and fine detail.
• Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language — A compact 135M parameter LLM for Bangla achieves 1B-tier performance by efficiently merging tokens, overcoming low-resource script challenges.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.