BioNLP at #SMM4H-HeaRD 2026 Task 3 Estimating Flu Vaccine Effectiveness: A Temporal-Aware Fine-Tuning and Similarity-Based Few-Shot Prompting Approach

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Biomedical Natural Language Processing · Depth: Expert, quick

Summary

Irina Patularu's systems for the SMM4H 2026 shared task addressed flu-related tweet classification across two subtasks: flu vaccination status and flu test outcome. The research evaluated two distinct approaches. The first involved fine-tuning BERTweet-large with a temporal-aware architecture, cross-validation ensembling, and regularization techniques. The second utilized a GPT-4o few-shot prompting system, incorporating similarity-based dynamic example retrieval, chain-of-thought reasoning, and contrastive label ranking. Fine-tuning proved superior for the flu vaccination subtask, achieving a micro-F1 of 87.90% with sufficient and relatively balanced training data. Conversely, few-shot prompting performed better for the flu test subtask, reaching a micro-F1 of 95.74%, particularly effective with limited and heavily imbalanced training data where fine-tuning was less effective.

Key takeaway

For NLP Engineers developing tweet classification systems, your choice between fine-tuning and few-shot prompting should hinge on your dataset's characteristics. If you possess sufficient and relatively balanced training data, fine-tuning models like BERTweet-large will likely yield superior performance. Conversely, when dealing with limited and heavily imbalanced data, few-shot prompting with large language models such as GPT-4o offers a more effective solution, achieving higher accuracy. Evaluate your data's quantity and balance early to optimize model selection and performance.

Key insights

Optimal NLP approach for tweet classification depends on training data quantity and balance.

Principles

• Fine-tuning benefits from ample, balanced datasets.
• Few-shot prompting thrives on scarce, imbalanced data.

Method

Fine-tuning BERTweet-large used temporal-aware architecture, cross-validation ensembling, and regularization. GPT-4o few-shot prompting employed similarity-based example retrieval, chain-of-thought, and contrastive label ranking.

In practice

• Apply fine-tuning for robust, data-rich tasks.
• Leverage few-shot prompting for data-scarce scenarios.

Topics

• BioNLP
• Social Media Mining
• Tweet Classification
• Flu Vaccine Effectiveness
• BERTweet-large
• GPT-4o
• Few-Shot Learning

Best for: AI Scientist, Research Scientist, NLP Engineer

Related on AIssential

• Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models — LLMs can effectively detect HIV-related stigma in clinical notes, with encoder models outperforming generative models in zero-shot settings.
• The Hardest Part of Fine-Tuning Isn’t the Training — The most challenging aspect of NLP fine-tuning is effective label design and meticulous data annotation, not the model training itself.
• RecallRisk-BERT: A Multi-Task Framework for Post-Report Medical Device Recall Triage — Multi-task text-tabular models effectively triage medical device recalls by jointly predicting severity and root-cause.
• An Evolutionary Approach for Designing Stable and Highly Expressible Low-Immunogenicity Therapeutic mRNA Sequences — The BERT-GA framework optimizes therapeutic mRNA sequences for balanced translation efficiency, structural stability, and low immunogenicity.
• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answering — Retrieved evidence can directly supervise query-side attention adaptation for improved long-context QA in smaller LLMs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.