Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Science & Research — Health & Medical Research, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

An empirical study published in BioNLP 2026 by Belmadani et al. investigates the effectiveness of Large Language Model (LLM) adaptation strategies for specialized domains and languages, focusing on French medical question answering (QA). The research compares continual pretraining (CPT), supervised fine-tuning (SFT), and their combination across three model families, multiple sizes, and three initialization types. For multiple-choice QA (MCQA), CPT+SFT frequently achieved the highest scores, though SFT alone proved a strong and cost-effective default due to minimal, often non-significant gains from CPT. In open-ended QA (OEQA), CPT consistently improved overlap-based metrics, while SFT often degraded generation quality, with instruction tuning and CPT+SFT preferred by LLM-based evaluation. The study also demonstrated effective cross-lingual transfer from French adaptation to English benchmarks, offering practical guidelines for strategy selection under computational constraints.

Key takeaway

For Machine Learning Engineers adapting LLMs for specialized medical domains, you should prioritize supervised fine-tuning (SFT) for multiple-choice QA tasks, as it offers a cost-effective solution with performance comparable to more complex CPT+SFT approaches. For open-ended QA, consider combining continual pretraining (CPT) with SFT or instruction tuning to improve generation quality. Your adaptation efforts in one language, like French, can effectively transfer to other languages, such as English, broadening the impact of your work.

Key insights

Domain adaptation strategies for medical LLMs show trade-offs between CPT, SFT, and their combination, varying by QA task type.

Principles

• SFT is a cost-effective default for MCQA.
• CPT improves OEQA overlap metrics.
• Cross-lingual transfer from domain adaptation is effective.

Method

The study compares CPT, SFT, and CPT+SFT across three LLM families, multiple sizes, and three initialization types, evaluating MCQA and OEQA with automatic metrics and LLM-as-a-Judge.

In practice

• Prioritize SFT for MCQA tasks with budget constraints.
• Combine CPT with SFT or instruction tuning for OEQA.
• Consider cross-lingual adaptation for resource-scarce languages.

Topics

• LLM Adaptation
• Medical QA
• Continual Pretraining
• Supervised Fine-tuning
• Cross-lingual Transfer
• French NLP

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA — SFT offers the best performance-efficiency trade-off for medical MCQA, while CPT benefits OEQA but with caveats.
• Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA — For medical LLM adaptation, SFT is cost-effective for MCQA, but CPT or CPT+SFT is preferred for OEQA, showing cross-lingual transfer.
• Teaching LLMs Brazilian Healthcare: Injecting Knowledge from Official Clinical Guidelines — Domain-specific synthetic data and reinforcement learning significantly enhance LLM performance on specialized clinical knowledge.
• Understanding post-training of LLMs: SFT — Post-training guides LLM behavior and expression, not knowledge, primarily via SFT and preference optimization.
• Domain-Adapted Small Language Models for Reliable Clinical Triage — Domain-adapted SLMs can provide reliable, privacy-preserving ESI decision support, outperforming larger proprietary models.
• Adapting AIM LLMs For Specific Use Cases Through Fine-Tuning in AMD AI Workbench — Fine-tuning adapts pre-trained LLMs for specific tasks, but core knowledge largely stems from pre-training.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.