Dr-BERT-NL at #SMM4H–HeaRD 2026: DOKTERBERT – Ontology-Grounded Contextual Representations for Dutch Clinical NLP

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research, Natural Language Processing · Depth: Expert, quick

Summary

Dr-BERT-NL submitted a system to SMM4H-HeaRD 2026 Task 7, focusing on labeling ClinicalImpacts and SocialImpacts spans within Reddit posts concerning non-medical substance use. The team compared four distinct pipeline architectures, all utilizing a DeBERTa-v3-base backbone. These included a direct 5-class encoder with a linear-chain CRF head, a two-stage detect-then-classify pipeline integrating instruction-tuned LLMs (Qwen2.5-7B or Gemma-3-12B, 4-bit NF4), an LLM-based audit pipeline verifying encoder predictions, and a classical-ML variant using an SVM on encoder span embeddings. Across 16 configurations, the encoder-only DeBERTa-v3 + CRF setup emerged as the strongest single system on the official test split, achieving 45.4% strict and 54.2% relaxed F1 scores. This performance represents a significant improvement of +8.6 and +5.3 points, respectively, over a mental-roberta-base baseline. Notably, LLM audits provided only a minor gain on the development set, which did not translate to the test set.

Key takeaway

For NLP engineers developing systems for clinical impact span detection, this research suggests that a direct DeBERTa-v3 + CRF architecture provides a strong baseline. You should prioritize optimizing such encoder-only models before integrating large language models, as LLM-based auditing or two-stage approaches may not yield superior performance on test data. Focus on robust, proven architectures for reliable span labeling.

Key insights

A direct encoder-CRF model outperformed LLM-integrated pipelines for clinical impact span labeling.

Principles

• Direct encoder-CRF architectures can be highly effective for span labeling.
• LLM auditing may not consistently transfer development gains to test performance.

Method

Four pipeline shapes were compared: direct encoder-CRF, two-stage LLM detect-then-classify, LLM audit of encoder predictions, and classical-ML SVM on encoder embeddings, all using DeBERTa-v3-base.

In practice

• Prioritize robust encoder-CRF models for clinical span detection.
• Evaluate LLM integration carefully for actual test set improvements.

Topics

• Clinical NLP
• Span Labeling
• DeBERTa-v3
• Conditional Random Fields
• Large Language Models
• Social Media Mining

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts — Domain-specific pre-training significantly enhances BERT models for Norwegian clinical NLP tasks.
• TalkTag: Fine-Grained Morphosyntactic Error Annotation for Transcribed Speech — TalkTag automates fine-grained morphosyntactic error annotation in spoken transcripts using an LLM, even with scarce data.
• Before the Labels: How Dataset Construction Shapes Suicidality Detection in Clinical Text — Dataset construction choices operationalize suicidality in EHRs, impacting NLP model reliability.
• EDEN: A Large-Scale Corpus of Clinical Notes for Italian — EDEN provides the largest Italian clinical note corpus and a CRF-filling benchmark for LLM-based information extraction.
• FineREX: Fine-Tuned NER-RE for Human Smuggling Knowledge Graphs — Domain-specific fine-tuning of LLMs significantly enhances knowledge graph construction quality and efficiency for complex, jargon-heavy legal texts.
• Evaluating FrameNet-Based Semantic Modeling for Gender-Based Violence Detection in Clinical Records — FrameNet-based semantic annotation significantly improves GBV detection in clinical text over categorical data alone.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.