zzucs at PsyDefDetect: Bridging Long-Tail Imbalance and Clinical Rubrics for DMRS Defense-Level Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, quick

Summary

The zzucs system, developed for the PsyDefDetect shared task at BioNLP 2026, addresses the challenge of detecting DMRS defense levels in emotional support dialogues. This system tackles severe class imbalance and fine-grained clinical distinctions by employing a data–supervision co-design strategy. Its components include SCCR, which uses stratified resampling to balance nine defense levels, improving macro-F1 by 4.9 points over random oversampling. CoR-QLoRA encodes clinical rubrics like task contracts and boundary cues into static prompts for 8B model fine-tuning. Submitted under sly_zzu, zzucs achieved a 0.3585 macro-F1 on the official blind-test leaderboard, ranking 6th among 21 teams and outperforming the strongest 8B baseline, Ministral-8B, by 4.4 F1 points.

Key takeaway

For NLP engineers developing models for clinically-nuanced text classification with severe class imbalance, consider integrating domain-specific rubrics via prompt engineering and employing stratified resampling. Your approach could significantly improve macro-F1 scores, as demonstrated by zzucs's 4.4 F1 point lead over 8B baselines in DMRS defense level detection. Explore data-supervision co-design to enhance model performance on challenging, fine-grained distinctions.

Key insights

Data-supervision co-design and clinical rubric encoding improve DMRS defense level detection in imbalanced datasets.

Principles

• Address class imbalance with stratified resampling.
• Encode clinical rubrics into model prompts.
• Combine data and supervision strategies.

Method

Apply stratified resampling (SCCR) for class balancing. Encode clinical rubrics (task contracts, taxonomy definitions, boundary cues) into static prompts for 8B model fine-tuning using CoR-QLoRA.

In practice

• Implement stratified resampling for imbalanced NLP tasks.
• Use prompt engineering to integrate domain rubrics.
• Fine-tune 8B models with specialized prompts.

Topics

• DMRS Defense Level Detection
• Class Imbalance
• Stratified Resampling
• Clinical Rubrics
• Prompt Engineering
• 8B Language Models
• BioNLP

Code references

• jackssdd/zzucs_psydefdetect_code

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Mitigating Data Scarcity in Psychological Defense Classification with Context-Aware Synthetic Augmentation — Context-aware synthetic data augmentation with theory-driven prompts significantly improves psychological defense mechanism classification in low-resource settings.
• Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation — Rubric-Conditioned Self-Distillation uses structured rubrics for fine-grained, token-level guidance in language model training, outperforming scalar rewards.
• Model-Agnostic Meta Learning for Class Imbalance Adaptation — HAMR adaptively reweights and resamples data to improve NLP model performance on imbalanced and difficult classes.
• A Reproducible Log-Driven AutoML Framework for Interpretable Pipeline Optimization in Healthcare Risk Prediction — Reproducible, log-driven AutoML reveals healthcare risk prediction pipeline component importance and search space redundancy.
• Data Presentation Over Architecture: Resampling Strategies for Credit Risk Prediction with Tabular Foundation Models — Context construction significantly impacts Tabular Foundation Model performance in imbalanced credit risk prediction.
• Exploring Data Augmentation and Resampling Strategies for Transformer-Based Models to Address Class Imbalance in AI Scoring of Scientific Explanations in NGSS Classroom — Targeted data augmentation significantly improves transformer-based model performance in automated scoring of imbalanced educational text data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.