Eraserhead at PsyDefDetect: Prompt Design and Class Rebalancing for Psychological Defense Mechanism Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Science & Research — Artificial Intelligence & Machine Learning, Mental Health & Psychological Support, Research Methodology & Innovation · Depth: Advanced, quick

Summary

The Eraserhead system, submitted to the PsyDefDetect shared task at BioNLP 2026, addresses psychological defense level detection. This system frames the problem as a nine-class utterance classification over supportive dialogue, utilizing Qwen3-14B. A key challenge is significant class imbalance, where High-Adaptive responses are much more frequent than minority classes, leading models to favor the majority. Eraserhead tackles this with clinically informed prompt design, per-label oversampling, and precise inference settings. The team iteratively adjusted oversampling targets based on error analysis and predicted label distributions. The final system achieved an an official macro F1 of 0.3418 on Leaderboard 1 and 0.3947 on Leaderboard 2, securing 7th place among 21 teams on both. Analysis revealed difficulties distinguishing Minor Image Distorting defenses from High-Adaptive responses and a persistent over-prediction of the majority class, underscoring the challenge of modeling psychological function from text.

Key takeaway

For NLP Engineers developing psychological text analysis systems, addressing class imbalance is critical. If you are working with imbalanced datasets like those for defense mechanism detection, you should implement iterative class rebalancing strategies, such as per-label oversampling, informed by error analysis. This approach, combined with domain-specific prompt design for models like Qwen3-14B, can significantly improve minority class performance, even if distinguishing subtle psychological nuances remains challenging.

Key insights

The Eraserhead system uses Qwen3-14B, prompt design, and class rebalancing to detect nine psychological defense mechanisms from text.

Principles

• Class imbalance significantly impacts minority class performance.
• Clinically informed prompt design improves detection.
• Iterative oversampling adjustment is crucial.

Method

The system uses Qwen3-14B for nine-class utterance classification, combining clinically informed prompt design, per-label oversampling, and careful inference settings. Oversampling targets were iteratively adjusted via error analysis.

In practice

• Apply Qwen3-14B for text classification.
• Implement per-label oversampling for imbalanced data.
• Use error analysis to refine rebalancing strategies.

Topics

• Psychological Defense Mechanisms
• Utterance Classification
• Qwen3-14B
• Class Imbalance
• Prompt Engineering
• BioNLP Shared Task

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• One prompt is not enough: Instruction Sensitivity Undermines Embedding Model Evaluation — Instruction embedding model evaluations are unreliable due to high sensitivity to prompt phrasing, invalidating single-prompt benchmarks.
• Implicit and Indirect: Detecting Face-threatening and Paired Actions in Asynchronous Online Conversations — Detecting face-threatening actions in asynchronous conversations requires handling ambiguity and multiple interpretations.
• Automatic Essay Scoring Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses — Deep-learning AES models are both overstable and oversensitive due to bag-of-words behavior and dataset biases.
• From Logistic Regression to GPT-2: Building a Complete Spam Detection & Sentiment Analysis Pipeline — For imbalanced text classification, prioritize F1-score and Precision-Recall AUC over accuracy and ROC-AUC.
• PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback — PsyScore unifies essay scoring and adaptive feedback using a shared psychometric latent ability representation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.