Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents

2026-05-22 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Advanced, extended

Summary

This study evaluates four text chunking strategies—Recursive, Khmer-Aware, Sentence-Based, and LLM-Based—within a Retrieval-Augmented Generation (RAG) framework for Khmer agricultural documents. Using the BGE-M3 multilingual embedding model and FAISS, performance was assessed via 5-fold cross-validation on 18 question–answer pairs, measuring Average Retrieval Score (L2 distance), Answer Relevance, Khmer Coverage, and Khmer Intersection over Union (Khmer IoU). The character-based Recursive chunking method with a 300-character size achieved the best results, demonstrating the lowest L2 distance (0.4295 ± 0.0461), highest Answer Relevance (0.8663 ± 0.0199), and highest Khmer IoU (0.6441 ± 0.0347). A paired t-test confirmed a statistically significant improvement over Sentence-Based chunking in L2 distance (p = 0.0121). These findings underscore the critical role of segmentation granularity and structural preservation for optimizing dense retrieval in low-resource languages like Khmer.

Key takeaway

For Machine Learning Engineers building RAG systems with low-resource languages or technical documents, your chunking strategy should prioritize structural preservation. The Recursive method, particularly with a 300-character window, significantly improves retrieval accuracy and semantic coherence by maintaining hierarchical context. You should implement and evaluate recursive chunking first, as it statistically outperforms sentence-based or linguistic approaches for these challenging document types.

Key insights

Structural preservation in text chunking significantly outperforms linguistic or semantic methods for effective retrieval in low-resource, technical documents.

Principles

• Document structure dictates optimal chunk granularity.
• Recursive chunking maintains "contextual bundles."
• Linguistic heuristics alone are insufficient for complex scripts.

Method

A pipeline involving data preparation, four chunking strategies, BGE-M3 embedding, FAISS indexing, retrieval, and 5-fold cross-validation with four metrics (L2 distance, Answer Relevance, Khmer Coverage, Khmer IoU).

In practice

• Prioritize Recursive (300-char) for hierarchical documents.
• Use LLM-Based chunking for storage optimization.
• Avoid Sentence-Based for procedural low-resource text.

Topics

• Text Chunking
• Retrieval-Augmented Generation
• Low-Resource Languages
• Khmer Language
• BGE-M3 Embedding
• FAISS

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents — Effective text chunking significantly improves RAG performance in low-resource, morphologically complex languages.
• Text Chunking Explained: Strategies, Overlap, and Semantic Splitting for Vector Search — Effective text chunking is crucial for RAG and semantic search, directly impacting retrieval precision and context relevance.
• Why Naive Chunking Breaks RAG, and What to Build Instead — Structure-aware parsing and multimodal retrieval techniques are crucial for effective RAG on complex documents.
• Semantic Chunking and the Entropy of Natural Language — A new statistical model explains natural language redundancy through hierarchical semantic chunking and predicts variable entropy rates.
• RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora — Existing RAG benchmarks misrepresent real-world performance due to high document redundancy and similarity in enterprise corpora.
• Structured Summaries for Retrieval-Augmented Generation in Portuguese-Language Consumer Complaints — Structured summaries significantly improve dense retrieval effectiveness in RAG systems for noisy, lengthy texts.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.