Multi-Task Instruction Tuning via Data Scheduling for Low-Resource Arabic SpeechLLMs

2026-01-18 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing, Speech Processing · Depth: Expert, short

Summary

A controlled study investigates multi-task instruction tuning for an Arabic-centric audio LLM across generative and discriminative tasks in resource-constrained settings. The research addresses challenges in adapting audio LLMs to linguistically complex, dialect-rich environments like Arabic-English. Generative tasks include automatic speech recognition (ASR) and speech/text summarization, while discriminative tasks cover dialect identification (DID) and speech emotion recognition (SER). The study introduces AraMega-SSum, the first Arabic speech summarization dataset for training and benchmarking. Four training strategies were compared: Uniform Mixing (UM), Task-Progressive Curriculum (TPC), Aligner-Based Diverse Sampling (ADS), and a two-stage TPC->ADS approach. TPC achieved the strongest performance on generative tasks. The two-stage TPC->ADS strategy provided the best overall balance, outperforming large proprietary models like Gemini-2.5-Pro on discriminative tasks and showing strong DID and SER performance. AraMega-SSum and all experimental resources will be publicly released.

Key takeaway

For Machine Learning Engineers developing audio LLMs for low-resource, dialect-rich languages like Arabic, consider a two-stage Task-Progressive Curriculum (TPC) followed by Aligner-Based Diverse Sampling (ADS). This strategy balances generative task performance, such as ASR and summarization, with discriminative tasks like dialect identification and emotion recognition. It offers a robust approach to overcome linguistic complexity and resource constraints, potentially outperforming larger proprietary models on specific tasks.

Key insights

Multi-task instruction tuning with data scheduling effectively adapts audio LLMs to low-resource, dialect-rich Arabic speech.

Principles

• Generative and discriminative tasks require different tuning strategies.
• Data scheduling impacts efficiency-robustness trade-offs in multi-task learning.
• Two-stage tuning can balance diverse task performance in complex linguistic settings.

Method

The study compared Uniform Mixing, Task-Progressive Curriculum, Aligner-Based Diverse Sampling, and a two-stage TPC->ADS strategy for multi-task instruction tuning of Arabic audio LLMs.

In practice

• Use TPC for strong generative task performance in audio LLMs.
• Combine TPC with ADS for balanced multi-task results across generative and discriminative tasks.
• Utilize the AraMega-SSum dataset for Arabic speech summarization training and benchmarking.

Topics

• Multi-Task Instruction Tuning
• Audio LLMs
• Arabic Speech Processing
• Data Scheduling
• Low-Resource NLP
• AraMega-SSum

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning — Multilingual code-switching instruction tuning improves LLM performance beyond bilingual settings.
• Low-resource Language Discrimination Towards Chinese Dialects with Transfer learning and Data Augmentation — Transfer learning and data augmentation effectively overcome low-resource challenges in Chinese dialect discrimination.
• Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language — A compact 135M parameter LLM for Bangla achieves 1B-tier performance by efficiently merging tokens, overcoming low-resource script challenges.
• Matching Tasks to Objectives: Fine-Tuning and Prompt-Tuning Strategies for Encoder-Decoder Pre-trained Language Models — Aligning task requirements with pre-training objectives significantly boosts encoder-decoder PLM performance.
• Efficient ASR Training with Conversations that Never Happened — LLM-generated synthetic conversations, synthesized via TTS, effectively augment scarce real data for ASR training.
• DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use — Scaling diversity in agentic task synthesis through evidence-first execution significantly improves LLM generalization for tool use.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.