PrionNER: A Named Entity Recognition Dataset for Prion Disease Biomedical Literature

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Science & Research — Health & Medical Research, Artificial Intelligence & Machine Learning · Depth: Advanced, quick

Summary

PrionNER is a newly released, manually annotated named entity recognition dataset designed for extracting clinical information related to prion diseases from PubMed abstracts. Comprising 317 abstracts, 2,943 sentences, and 6,955 text-bound entity annotations, the dataset covers 15 coarse-grained and 31 fine-grained clinically oriented entity types, including diseases, symptoms, diagnostics, and treatments. Its inter-annotator agreement reached 81.78 exact-match F1, indicating strong consistency. Benchmarking efforts showed W2NER as the strongest supervised model and Gemma-4-31B as the strongest zero-shot model, though the task remains challenging for complex mentions and subtle label distinctions. PrionNER aims to provide a clinically grounded benchmark for information extraction in rare-disease biomedical NLP, particularly under low-resource and fine-grained conditions.

Key takeaway

For Machine Learning Engineers developing NLP solutions for rare diseases, PrionNER offers a crucial, publicly available dataset to advance information extraction for prion diseases. You should integrate this dataset to train and evaluate your named entity recognition models, particularly when tackling fine-grained clinical entities. Utilize the provided benchmarks, noting that current models like W2NER and Gemma-4-31B still face challenges with complex mentions, indicating clear areas for your model improvement.

Key insights

PrionNER provides a critical, manually annotated NER dataset for extracting fine-grained clinical information on rare prion diseases from biomedical literature.

Principles

• Manual annotation ensures high clinical relevance.
• Specialized datasets are crucial for rare disease NLP.
• Fine-grained distinctions challenge current NER models.

Method

PubMed abstracts were manually annotated for 15 coarse-grained and 31 fine-grained clinical entity types, then benchmarked with supervised and zero-shot NER models.

In practice

• Utilize PrionNER for developing prion disease NER systems.
• Benchmark new models against W2NER and Gemma-4-31B.
• Address challenges in complex entity recognition.

Topics

• Named Entity Recognition
• Prion Diseases
• Biomedical NLP
• Rare Disease NLP
• PrionNER Dataset
• Information Extraction

Code references

• daotuanan/PrionNER

Best for: NLP Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Nested Named Entity Recognition in Plasma Physics Research Articles — A lightweight BERT-CRF model with entity-specific specialization and Bayesian Optimization effectively extracts nested entities from plasma physics texts.
• GERNERMED++: Semantic annotation in German medical NLP through transfer-learning, translation and word alignment — GERNERMED++'s entity recognition model uses spaCy's transducer-based parser with a BILOU scheme.
• AAbAAC: An Annotated Corpus for Autoimmunity Information Extraction — Small, specialized annotated corpora significantly boost information extraction performance in niche biomedical domains.
• How to Leverage NER and Advanced NLP Techniques for Life Sciences — NER and advanced NLP are essential for structuring and interpreting the vast unstructured text data in Life Sciences.
• KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts — Domain-specific pre-training significantly enhances BERT models for Norwegian clinical NLP tasks.
• spaCy’s entity recognition model: incremental parsing with Bloom embeddings & residual CNNs — spaCy v2's NER model leverages transition-based parsing, Bloom embeddings, and residual CNNs, underscoring the need for domain-specific training data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.