Team Gazoo! at #SMM4H-HeaRD 2026: Zero-Training NER via Iterative LLM Prompt Self-Optimization for Opioid Impact Span Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Health & Medical Research · Depth: Expert, quick

Summary

Team Gazoo!'s system for the #SMM4H-HeaRD 2026 shared task achieved top rankings in detecting self-reported clinical and social impacts of nonmedical opioid use in social media text. Their zero-training, prompt-only approach utilizes a GPT-5.4 large language model with structured few-shot prompting and autonomous, iterative rule optimization. The system integrates a domain-specific entity ontology, three core decision rules, and 65 cognitively organized few-shot examples into a single prompt, with BIO constraint post-processing. Crucially, the LLM refined its own prompt by analyzing errors and proposing targeted edits to its rules and examples over 18 self-refinement cycles. This method yielded an F1-Strict of 0.53 and F1-Relaxed of 0.60 on the test set, securing first place among all participating teams.

Key takeaway

For NLP Engineers developing Named Entity Recognition systems with limited training data, this work demonstrates that iterative LLM prompt self-optimization can achieve leading performance. You should explore integrating autonomous error analysis and prompt refinement into your LLM-based NER workflows to significantly improve accuracy without extensive manual tuning or dataset creation. Consider structuring prompts with domain ontologies and few-shot examples.

Key insights

Iterative LLM prompt self-optimization enables zero-training Named Entity Recognition with superior performance.

Principles

• LLMs can autonomously refine their own prompts by analyzing errors.
• Structured few-shot prompting with domain ontology enhances zero-training NER.

Method

An LLM analyzes its own output errors and iteratively proposes targeted edits to its internal rules and few-shot examples over multiple refinement cycles, enhancing performance.

In practice

• Apply LLM self-optimization for zero-training NER in specialized domains.
• Encode domain-specific ontologies and examples directly into LLM prompts.

Topics

• Named Entity Recognition
• Large Language Models
• Prompt Engineering
• Zero-Shot Learning
• Opioid Impact Detection
• Self-Optimization

Best for: AI Engineer, Machine Learning Engineer, AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents — Prompt engineering with LLMs offers a viable alternative for legal NER in low-resource settings.
• DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition — Simulating human pilot annotation with LLMs to refine instructions significantly boosts zero-shot NER performance.
• Analysing Lightweight Large Language Models for Biomedical Named Entity Recognition on Diverse Ouput Formats — Lightweight LLMs offer competitive biomedical NER performance, with specific output formats proving more effective than multi-format tuning.
• Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification — In-context learning for Turkish LVCs is highly prompt-sensitive, with few-shot demonstrations improving LLM calibration to match supervised baselines.
• On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance — LLM annotation reliability hinges on definition alignment, not just text memorization, as prompt-based corrections are largely ineffective.
• SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection — Careful prompt design with large-scale augmentation effectively injects knowledge into LLMs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.