No_gmail at #SMM4H-HeaRD 2026: Detecting Patient Metadata in COVID-19 Scientific Literature: A Comparative Study of Encoder-Only and Autoregressive Language Models

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Science & Research — Health & Medical Research, Mathematics & Computational Sciences · Depth: Advanced, quick

Summary

A comparative study by No_gmail at #SMM4H-HeaRD 2026 investigated methods for detecting patient metadata within COVID-19 scientific literature, a crucial step for genomic epidemiology that typically requires extensive manual curation. The research compared fine-tuned encoder-only models, specifically BERT and BioLinkBERT, against autoregressive large language models including Llama, Gemma, and GPT-OSS. Both prompting and fine-tuning regimes were evaluated, with techniques like Focal Loss and undersampling employed to manage severe class imbalance. The study found that encoder-only models substantially outperformed their autoregressive counterparts. BioLinkBERT-base, utilizing Focal Loss, achieved a macro F1 score of 0.76, significantly higher than the 0.54 attained by the best fine-tuned autoregressive model.

Key takeaway

For machine learning engineers developing NLP solutions for biomedical text analysis, particularly for patient metadata detection, you should prioritize encoder-only architectures. This study demonstrates their superior performance, with BioLinkBERT-base achieving a macro F1 of 0.76 compared to 0.54 for autoregressive models. When facing severe class imbalance, integrate techniques like Focal Loss and undersampling into your training pipeline to optimize model effectiveness.

Key insights

Encoder-only models significantly outperform autoregressive LLMs for patient metadata detection in highly imbalanced biomedical text.

Principles

• Encoder-only models excel in specific text classification tasks.
• Class imbalance requires specialized loss functions and sampling.

Method

Fine-tuning encoder-only models (BERT, BioLinkBERT) and autoregressive LLMs (Llama, Gemma, GPT-OSS) with prompting, Focal Loss, and undersampling to identify patient metadata sentences.

In practice

• Prioritize encoder-only architectures for biomedical text classification.
• Implement Focal Loss and undersampling for severe class imbalance.

Topics

• COVID-19 Literature
• Patient Metadata Detection
• Encoder-Only Models
• Autoregressive LLMs
• Class Imbalance
• Genomic Epidemiology
• Natural Language Processing

Best for: NLP Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models — LLMs can effectively detect HIV-related stigma in clinical notes, with encoder models outperforming generative models in zero-shot settings.
• MetaDent: Labeling Clinical Images for Vision-Language Models in Dentistry — MetaDent provides a novel dataset and benchmarks to advance VLM capabilities in fine-grained intraoral image analysis.
• Field of Science and Technology Classification of Academic Documents in Portuguese — Fine-tuned encoder models outperform general LLMs for FOS classification of Portuguese academic documents.
• AAbAAC: An Annotated Corpus for Autoimmunity Information Extraction — Small, specialized annotated corpora significantly boost information extraction performance in niche biomedical domains.
• Meta-encoder: a unified integration framework for multiple pathological foundation models in cancer detection — The Meta-Encoder integrates multiple pathological foundation models to improve cancer detection and molecular characterization.
• Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM — Combining sparse lexical features with dense semantic embeddings improves clinical dosing error detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.