Forgotten Words: Benchmarking NeoBERT for Dementia Detection in Low-Resource Conversational Filipino and English Speech

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, short

Summary

A systematic evaluation benchmarks transformer-based models for dementia detection in low-resource conversational Filipino and English speech, addressing the English-centric bias in NLP systems. Researchers constructed a parallel bilingual dataset of 4,000 DementiaBank-derived transcripts, manually translated into Filipino to preserve discourse markers. Five model families—TF-IDF + LogReg, BERT, NeoBERT, XLM-R, and RoBERTa-Tagalog—were assessed under monolingual, zero-shot cross-lingual, and bilingual fine-tuning settings. Key findings indicate that in-domain performance does not transfer across languages, with English-trained BERT dropping to Macro-F1 = 0.455 on Filipino. Architectural modernization alone did not improve robustness. However, bilingual fine-tuning effectively eliminated cross-lingual degradation across all transformer models, achieving Macro-F1 scores of 0.969–0.973. These results highlight that multilingual clinical NLP performance is primarily driven by linguistic coverage during training, rather than model scale or architecture.

Key takeaway

For NLP Engineers developing clinical applications in multilingual settings, especially those involving code-switching, you should prioritize creating or acquiring parallel bilingual datasets for fine-tuning. Relying on monolingual models or architectural upgrades alone will likely result in significant performance degradation, as English-trained BERT showed a drop to Macro-F1 = 0.455 on Filipino. Instead, implement bilingual fine-tuning to achieve high cross-lingual performance, as demonstrated by Macro-F1 scores of 0.969–0.973.

Key insights

Bilingual fine-tuning is crucial for robust cross-lingual dementia detection in low-resource languages.

Principles

• Linguistic coverage drives multilingual clinical NLP performance.
• Architectural modernization alone does not ensure cross-lingual robustness.
• In-domain performance does not transfer across languages.

Method

Construct a parallel bilingual dataset, manually translating to preserve discourse markers, then apply bilingual fine-tuning.

In practice

• Develop bilingual datasets for underrepresented languages.
• Prioritize linguistic diversity in clinical NLP model training.
• Evaluate cross-lingual transfer before deployment.

Topics

• Dementia Detection
• Clinical NLP
• Transformer Models
• Bilingual Fine-tuning
• Filipino Language
• Low-Resource NLP

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Multilingual Detection of Alzheimer's Disease from Speech: A Cross-Linguistic Transfer Learning Approach — Cross-language transfer learning effectively detects Alzheimer's Disease from speech across multiple languages, achieving strong generalization and rapid inference.
• Improving low-resource ASR using bilingual fine-tuning with language identification: a cross-linguistic evaluation — Bilingual fine-tuning enhances low-resource ASR, particularly when language identification is accurate or explicitly guided during inference.
• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts — Domain-specific pre-training significantly enhances BERT models for Norwegian clinical NLP tasks.
• Creating Multilingual Mental Health Dialogue Datasets: Limits of Persona-Based Localization via Nationality and Language — Persona-based localization via nationality and language is insufficient for generating consistent multilingual mental health datasets.
• Global vs. Local Sentence Embeddings for Brazilian Portuguese: Revisiting Monolingual Models in the Age of Foundation Models — Optimal model choice for PT-BR sentence embeddings balances vocabulary coverage and adaptation flexibility.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.