PEI at #SMM4H-HeaRD 2026: Enhancing Patient Metadata Detection via Hypothesis-Conditioned Classification and Paraphrase-Based Data Augmentation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Health & Wellbeing — Health & Medical Research, Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

PEI presented its approach to Task 5 of the #SMM4H-HeaRD 2026 Workshop, focusing on binary classification for detecting patient metadata in SARS-CoV-2 sequencing articles. The team explored both encoder-based BioM-BERT as a baseline and the large language model Mistral-Nemo. To enhance performance, a paraphrase-based data augmentation pipeline utilizing Qwen3 was implemented, adding paraphrased training and validation instances for fine-tuning. For Mistral-Nemo, prompt refinement and error analysis were conducted, while the BioM-BERT model adopted a hypothesis-conditioned classification task inspired by Natural Language Inference. These methods significantly improved both models: Mistral-Nemo's F1 score rose from 0.423 to 0.750, and BioM-BERT's F1 score increased from 0.801 to 0.821 on the validation set. The best BioM-BERT model achieved an F1-score of 0.786 on the test set, surpassing the mean and median of competing systems. The best-performing model is released on Hugging Face for reproducibility.

Key takeaway

For NLP Engineers developing medical text classification systems, you should consider integrating paraphrase-based data augmentation with models like BioM-BERT or Mistral-Nemo. Specifically, reformulating binary classification as a hypothesis-conditioned task can boost encoder performance, as demonstrated by the 0.786 F1-score on the #SMM4H-HeaRD 2026 test set. Additionally, utilizing LLMs for data augmentation, such as Qwen3 for paraphrasing, can significantly improve fine-tuning, offering a robust strategy for enhancing detection accuracy and reproducibility.

Key insights

Combining paraphrase-based data augmentation and NLI-inspired classification significantly improves patient metadata detection.

Principles

• Data augmentation via paraphrasing enhances model fine-tuning.
• NLI-inspired task reformulation benefits encoder models.
• LLMs require prompt refinement and error analysis.

Method

The approach involves paraphrase-based data augmentation using Qwen3, prompt refinement for LLMs like Mistral-Nemo, and reformulating encoder-based tasks (BioM-BERT) as hypothesis-conditioned classification.

In practice

• Apply Qwen3 for paraphrase-based data augmentation.
• Reformulate binary classification as NLI for encoder models.
• Release best models on Hugging Face for community use.

Topics

• Patient Metadata Detection
• Natural Language Processing
• Data Augmentation
• Large Language Models
• BioM-BERT
• Hypothesis-Conditioned Classification
• SARS-CoV-2

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Data Augmentation for Historical NER: A Systematic Comparison of Lexical and LLM-based Approaches — This paper systematically compares lexical and LLM-based data augmentation for Historical Named Entity Recognition.
• Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models — LLMs can effectively detect HIV-related stigma in clinical notes, with encoder models outperforming generative models in zero-shot settings.
• EDEN: A Large-Scale Corpus of Clinical Notes for Italian — EDEN provides the largest Italian clinical note corpus and a CRF-filling benchmark for LLM-based information extraction.
• Improving Answer Extraction in Context-based Question Answering Systems Using LLMs — Fine-tuning pre-trained LLMs on benchmark QA datasets significantly improves answer extraction accuracy and contextual understanding in QA systems.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Identifying Signs and Symptoms of Urinary Tract Infection from Emergency Department Clinical Notes Using Large Language Models — Prodigy is an efficient, scriptable annotation tool enabling data scientists to accelerate NLP development.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.