NoviceTrio in #SMM4H-HeaRD 2026: Hybrid Clinical Transformer Ensembles for Insomnia Detection and Evidence Extraction from Clinical Notes

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Advanced, quick

Summary

NoviceTrio developed two systems for the #SMM4H-HeaRD 2026 Task 2, focusing on automated insomnia detection and evidence extraction from clinical notes. Subtask 1, binary insomnia classification, employs an ensemble of Qwen3-4B-Instruct and Bio_ClinicalBERT, utilizing chunk-based processing with overlapping token windows for long notes. Subtask 2 involves a dual-head multi-task transformer model for multi-label rule prediction and token-level evidence span extraction using BIO tagging. This system also incorporates sentence-level filtering via sentence-transformer embeddings for clinical relevance. The Subtask 1 system achieved a 0.9474 recall, surpassing shared-task mean and median. Subtask 2 also exceeded mean and median scores across classification, exact match, and partial match metrics. The end-to-end implementation is publicly available on GitHub.

Key takeaway

For NLP engineers developing clinical systems, especially for classification and evidence extraction from extensive clinical notes, consider hybrid transformer ensembles. Combining models like Qwen3-4B-Instruct and Bio_ClinicalBERT, alongside multi-task learning and chunk-based processing, demonstrates competitive performance. You should explore these architectural patterns and the publicly available implementation to enhance accuracy and efficiency in similar domain-specific tasks.

Key insights

Hybrid clinical transformer ensembles effectively detect insomnia and extract evidence from clinical notes.

Principles

• Combine general and domain-specific models
• Use chunking for long text processing
• Employ multi-task learning for related predictions

Method

An ensemble of Qwen3-4B-Instruct and Bio_ClinicalBERT handles classification, while a dual-head multi-task transformer with BIO tagging performs evidence extraction, enhanced by sentence-level filtering.

In practice

• Detect insomnia from clinical notes
• Extract specific evidence spans
• Process lengthy medical documents

Topics

• Insomnia Detection
• Clinical NLP
• Transformer Ensembles
• Evidence Extraction
• Qwen3-4B-Instruct
• Bio_ClinicalBERT
• Multi-task Learning

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM — Combining sparse lexical features with dense semantic embeddings improves clinical dosing error detection.
• Multilingual Detection of Alzheimer's Disease from Speech: A Cross-Linguistic Transfer Learning Approach — Cross-language transfer learning effectively detects Alzheimer's Disease from speech across multiple languages, achieving strong generalization and rapid inference.
• Language Models as Interfaces, Not Oracles: A Hybrid LLM-ML System for Pediatric Appendicitis — Hybrid AI systems improve clinical decision support by using LLMs for feature extraction and validated ML models for reliable, auditable prediction.
• Noise reduction in BERT NER models for clinical entity extraction — A post-processing Noise Removal model significantly boosts clinical NER precision by analyzing probability distributions and contextual features.
• EDEN: A Large-Scale Corpus of Clinical Notes for Italian — EDEN provides the largest Italian clinical note corpus and a CRF-filling benchmark for LLM-based information extraction.
• Multi-Stage Training for Abusive Comment Detection in Indic Languages — A multi-stage pipeline combining language preprocessing and model ensembles effectively detects abusive Indic language comments while minimizing false positives.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.