BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

2026-07-06 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

BaFCo is a new benchmark dataset designed to address the scarcity of high-quality annotated data for document comprehension in low-resource languages, specifically Bangla. It comprises 200 multi-page complex Bangladeshi government forms from diverse sectors like agriculture, education, banking, and land management. The dataset features a fine-grained annotation schema with 26 types of form entities and a coarse set with 5 types, capturing structural and contextual complexity for Document Layout Analysis (DLA) and Key Information Extraction (KIE). Evaluations of leading Multimodal Large Language Models (MLLMs) from the ChatGPT, Gemini, Claude, Qwen, and Kimi series, using zero-shot and chain-of-thought prompts under low and high reasoning setups, revealed significant limitations in their ability to comprehend Bangla forms, particularly in localizing highly granular entities. The dataset and code are publicly available.

Key takeaway

For NLP Engineers and AI Scientists developing MLLMs for global applications, this research highlights a critical need. You must improve performance on low-resource languages and complex document structures. Prioritize developing models capable of accurately localizing highly granular form entities in languages like Bangla. Consider utilizing the BaFCo dataset to benchmark your models' capabilities and drive targeted improvements for real-world, human-centric applications.

Key insights

Current MLLMs struggle with complex document understanding in low-resource languages like Bangla, highlighting a critical data and model gap.

Principles

• Low-resource languages lack quality annotated data.
• Complex forms require fine-grained entity annotation.
• MLLMs show limitations in granular entity localization.

Method

The BaFCo method involves curating 200 multi-page government forms, defining 26 fine-grained and 5 coarse form entity types, and evaluating MLLMs with zero-shot and chain-of-thought prompts under varying reasoning setups.

In practice

• Use BaFCo to benchmark MLLMs for Bangla.
• Develop MLLMs specifically for low-resource forms.
• Focus on granular entity localization.

Topics

• Document Understanding
• Multimodal Large Language Models
• Low-Resource Languages
• Bangla Language
• Benchmark Datasets
• Key Information Extraction
• Document Layout Analysis

Best for: Research Scientist, AI Scientist, NLP Engineer, Computer Vision Engineer

Related on AIssential

• Bangla Key2Text: Text Generation from Keywords for a Low Resource Language — A new 2.6M Bangla keyword-text dataset enables supervised keyword-driven text generation for low-resource languages.
• Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning — Scan-oriented reasoning benchmarks reveal MLLM deficiencies in full-document understanding and cross-checking.
• PorTEXTO: A European Portuguese Benchmark for Visual Text Extraction — Specialized multilingual data significantly improves European Portuguese OCR performance over model size or resolution.
• Annotation Challenges in Low-Resource African Languages — High-quality NLP for African languages demands culturally-attuned annotation, robust QA, and fair, community-centric practices.
• Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language — A compact 135M parameter LLM for Bangla achieves 1B-tier performance by efficiently merging tokens, overcoming low-resource script challenges.
• Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents — Structural preservation in text chunking significantly outperforms linguistic or semantic methods for effective retrieval in low-resource, technical documents.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.