LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features

2026-07-23 · Source: cs.CL updates on arXiv.org · Field: Science & Research — Health & Medical Research, Mathematics & Computational Sciences · Depth: Expert, long

Summary

A novel framework for dementia detection utilizes a low-rank adaptation (LoRA)-tuned large language model (LLM) to perform structured multi-view reasoning over speech-derived features. Developed by NAVER Cloud and Ewha Womans University, this approach unifies four complementary signals: ASR transcripts with pause markers, discourse-level topic cues, temporal fluency statistics, and phonological sequences, all encoded within a single JSON-structured prompt. The system achieved an F1-score of 90.14% on the ADReSSo challenge dataset, outperforming prior systems. Ablation studies confirmed the incremental diagnostic value of each view, with discourse-level cues contributing the largest performance gain. This method leverages LLMs like Qwen3 and Gemma-3, adapted with LoRA (r=8, α=16), to learn a coherent decision function without requiring modality-specific encoders or late-stage fusion.

Key takeaway

For machine learning engineers developing speech-based dementia detection systems, you should consider unifying diverse speech features into a single structured prompt for LLM fine-tuning. This approach, using LoRA, simplifies the architecture by avoiding separate encoders and late-stage fusion, potentially improving diagnostic accuracy. You can incorporate robust ASR, temporal fluency, phonological, and discourse cues to enhance model performance, especially discourse-level information.

Key insights

LoRA-tuned LLMs can effectively detect dementia by integrating diverse speech features via a unified structured prompt, achieving high F1-scores.

Principles

• Dementia manifests across complementary speech dimensions.
• Unified prompt encoding enables integrative LLM reasoning.
• Model capacity scales with performance for multi-view cues.

Method

A LoRA-adapted LLM processes a JSON-structured prompt containing ASR transcripts, pause markers, discourse topics, temporal fluency, and phonological sequences. This enables joint reasoning for binary dementia classification.

In practice

• Encode diverse speech features into a single JSON prompt.
• Use LoRA for efficient LLM adaptation on limited datasets.
• Leverage Whisper for robust ASR transcripts.

Topics

• Dementia Detection
• Large Language Models
• LoRA Fine-tuning
• Multi-View Speech Features
• ADReSSo Benchmark
• Prompt Engineering

Code references

• vivivic/is26dementia

Best for: AI Engineer, NLP Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• VITECH@DravidianLangTech2026: Prompting and LoRA Adaptation for Tamil Abusive Language Detection - A Comparative Study of Open LLMs — LoRA fine-tuning significantly improves open LLM performance for Tamil abusive language detection over prompting.
• Noise-Aware Visual Representation Learning for Medical Visual Question Answering — Learning robust visual representations through denoising enhances Medical Visual Question Answering performance and robustness.
• Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting — LoRA fine-tuning can cause significant forgetting, which a new output space regularizer effectively mitigates without replay data.
• JCT 2026 - SemEval Task 5 — Combining a generative LLM with a specialized cross-encoder enhances performance on complex semantic tasks like NLI and similarity.
• Listening Between the Lines: Joint Learning of ASR Embeddings and LLM-Augmented Linguistics for Dementia Detection — Jointly learning ASR embeddings and LLM-augmented linguistics significantly improves dementia detection from speech.
• LoRA & QLoRA Mastery: The Beginner-to-Advanced Guide to Efficient LLM Fine-Tuning — LoRA and QLoRA enable efficient LLM fine-tuning by adapting a small fraction of parameters, drastically reducing memory and compute.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.