LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Clinical Natural Language Processing · Depth: Expert, quick

Summary

The LinguIUTics team achieved a macro F1-score of 0.3917 in the PsyDefDetect 2026 shared task, ranking 4th among 21 teams for 9-class psychological defense mechanism classification. This performance represents a +7.7 absolute point (+24.4% relative) improvement over the Ministral-8B baseline (31.48 macro F1). Addressing severe class imbalance that hindered BERT-family encoders and zero-shot LLMs on rare classes, the team employed QLoRA fine-tuning of Qwen3-8B. Their methodology incorporated three key strategies: grouped stratified cross-validation to prevent data leakage, minority-class round-robin lexical augmentation, and a post-processing pipeline featuring logitbias tuning and ensemble blending. These components significantly enhanced minority-class recall, notably boosting the critical "Unclear" class (Level 8) from near-zero performance to an F1-score of 0.797.

Key takeaway

For clinical NLP engineers developing models for psychological defense mechanism classification or similar imbalanced text tasks, your strategy must explicitly address rare classes. Standard BERT-family or zero-shot LLM approaches are insufficient. You should consider QLoRA fine-tuning of models like Qwen3-8B, integrating grouped stratified cross-validation, minority-class lexical augmentation, and a post-processing pipeline with logitbias tuning and ensemble blending to significantly improve minority-class recall and overall macro F1.

Key insights

QLoRA fine-tuning of Qwen3-8B with imbalance-aware strategies effectively classifies psychological defense mechanisms, boosting rare class performance.

Principles

• Class imbalance degrades rare class NLP performance.
• Grouped stratified cross-validation prevents data leakage.
• Augmentation and post-processing boost minority recall.

Method

QLoRA fine-tuning Qwen3-8B, using grouped stratified cross-validation, minority-class round-robin lexical augmentation, and a post-processing pipeline with logitbias tuning and ensemble blending.

In practice

• Fine-tune LLMs with QLoRA for imbalanced text.
• Employ grouped stratified cross-validation.
• Use lexical augmentation for rare classes.

Topics

• Clinical NLP
• Psychological Defense Mechanisms
• Qwen3-8B
• QLoRA Fine-tuning
• Class Imbalance
• Data Augmentation
• Ensemble Methods

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Mitigating Data Scarcity in Psychological Defense Classification with Context-Aware Synthetic Augmentation — Context-aware synthetic data augmentation with theory-driven prompts significantly improves psychological defense mechanism classification in low-resource settings.
• Complex & Intelligent Systems, Volume 12, Issue 3, March 2026 — The issue highlights advancements in AI, optimization, and security across diverse applications and system complexities.
• Multi-Stage Training for Abusive Comment Detection in Indic Languages — A multi-stage pipeline combining language preprocessing and model ensembles effectively detects abusive Indic language comments while minimizing false positives.
• SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization — SURGELLM enhances multi-task NLP performance by integrating task-aware feature gating and class-balanced normalization within a unified transformer framework.
• Model-Agnostic Meta Learning for Class Imbalance Adaptation — HAMR dynamically addresses class imbalance and data difficulty in NLP via adaptive weighting and neighborhood-aware resampling.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.