ModernBERT is more efficient than conventional BERT for chest CT findings classification in Japanese radiology reports

2026-04-03 · Source: Machine learning : nature.com subject feeds · Field: Science & Research — Health & Medical Research, Mathematics & Computational Sciences · Depth: Expert, medium

Summary

A study published in Scientific Reports on April 3, 2026, compared three Japanese language models—BERT Base, JMedRoBERTa, and ModernBERT—for multi-label classification of 18 chest CT findings from radiology reports. Researchers fine-tuned all models under identical conditions using the CT-RATE-JPN dataset. ModernBERT demonstrated superior efficiency, generating significantly fewer tokens and achieving faster training and inference times while maintaining comparable in-domain performance (74.7% exact match accuracy vs. 72.7% for BERT Base). However, when tested on an external, domain-shifted dataset called RR-Findings, ModernBERT showed the largest decline in exact match accuracy, with BERT Base outperforming both JMedRoBERTa and ModernBERT. Despite this, ModernBERT retained reasonable ranking ability, indicated by smaller average precision differences. The study highlights ModernBERT's computational advantages for in-domain tasks but underscores its sensitivity to linguistic variability in real-world clinical data.

Key takeaway

For NLP Engineers developing solutions for Japanese radiology reports, consider ModernBERT for its efficiency in token generation and faster training/inference. However, if your application involves diverse or domain-shifted real-world clinical data, you should prioritize extensive and varied training data or implement domain-specific calibration strategies to mitigate performance degradation and ensure robust deployment in heterogeneous clinical environments.

Key insights

ModernBERT offers computational efficiency for Japanese medical text classification but requires diverse data for robustness.

Principles

• Efficiency does not always equate to generalizability.
• Domain shift significantly impacts model performance.

Method

Three Japanese language models (BERT Base, JMedRoBERTa, ModernBERT) were fine-tuned on the CT-RATE-JPN dataset for multi-label classification of 18 chest CT findings, then evaluated on an internal test set and an external, domain-shifted RR-Findings dataset.

In practice

• Use ModernBERT for in-domain Japanese medical NLP.
• Prioritize diverse training data for clinical deployment.

Topics

• ModernBERT
• Japanese Radiology Reports
• Chest CT Classification
• Natural Language Processing
• Transformer Models

Best for: NLP Engineer, AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts — Domain-specific pre-training significantly enhances BERT models for Norwegian clinical NLP tasks.
• ModernBERT - Modern Replacement for BERT | RAG, Embeddings, Classification, Reranking — ModernBERT significantly advances encoder-only models with architectural innovations and massive training data, improving performance and context.
• Comparing YAKE! and KeyBERT for Next-Gen Keyword Extraction — KeyBERT offers semantic depth for keyword extraction, while YAKE! prioritizes speed and statistical efficiency.
• Spark NLP 6.3.3: ModernBERT Embeddings, Vector DB Integration, and Layout-Aware Document Processing — Spark NLP 6.3.3 significantly boosts NLP pipeline efficiency and multimodal document processing capabilities.
• NLP-based Page Classification for Efficient LLM Extraction from Brazilian Public Tender Documents — Combining NLP page classification with LLM extraction significantly reduces token count and cost for document processing.
• Disaster Question Answering with LoRA Efficiency and Accurate End Position — A LoRA-optimized Japanese BERT-Bi-LSTM hybrid system delivers high accuracy and efficiency for disaster-specific question answering.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.