DAL Team at PsyDefDetect: From Supervised Encoders to Hierarchical LLM-RAG for Psychological Defense Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Science & Research — Life Sciences & Biology, Health & Medical Research, Research Methodology & Innovation · Depth: Expert, short

Summary

The DAL Team at PsyDefDetect proposes a hierarchical framework for detecting psychological defense mechanisms within multi-turn dialogues. This system integrates large language models (LLMs), retrieval-augmented generation (RAG), and heuristic calibration. The approach breaks down prediction into coarse-to-fine reasoning stages, incorporating dialogue reconstruction, explanation-enhanced retrieval, and a hybrid LLM–supervised filtering mechanism. These components are designed to tackle severe label imbalance and the complexities of implicit, context-dependent labeling. Experiments conducted on the PsyDefDetect dataset demonstrate that the LLM-based RAG significantly improves performance on minority and ambiguous classes, achieving a Macro F1 score of 0.31. However, the research also highlights ongoing difficulties in the fine-grained discrimination of latent psychological constructs.

Key takeaway

For NLP Engineers developing dialogue systems for mental health or psychological analysis, this research suggests that integrating hierarchical LLM-RAG can significantly enhance the detection of subtle psychological defense mechanisms. You should consider decomposing complex classification tasks into coarse-to-fine stages and employing hybrid LLM–supervised filtering to improve performance on underrepresented or ambiguous classes, as demonstrated by the 0.31 Macro F1 score. This approach offers a robust method for handling context-dependent and imbalanced labeling challenges.

Key insights

A hierarchical LLM-RAG framework improves psychological defense detection in dialogues, especially for minority classes.

Principles

• Decompose complex predictions into stages.
• Address label imbalance with hybrid filtering.
• Contextualize labels via dialogue reconstruction.

Method

The framework uses coarse-to-fine reasoning, dialogue reconstruction, explanation-enhanced retrieval, and hybrid LLM–supervised filtering to detect psychological defense mechanisms in multi-turn dialogues.

In practice

• Apply LLM-RAG for imbalanced NLP tasks.
• Use dialogue reconstruction for context.
• Implement coarse-to-fine reasoning stages.

Topics

• Psychological Defense Mechanisms
• Large Language Models
• Retrieval-Augmented Generation
• Multi-turn Dialogue Systems
• BioNLP
• Label Imbalance

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Multi-View Attention Multiple-Instance Learning Enhanced by LLM Reasoning for Cognitive Distortion Detection — Combining LLMs with MIL and ELB components improves cognitive distortion detection by enhancing interpretability and reasoning.
• Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection — A mechanism-oriented taxonomy of indirect linguistic encoding significantly enhances LLM-based coded language detection.
• Mitigating Data Scarcity in Psychological Defense Classification with Context-Aware Synthetic Augmentation — Context-aware synthetic data augmentation with theory-driven prompts significantly improves psychological defense mechanism classification in low-resource settings.
• Emergence of Hierarchical Emotion Organization in Large Language Models — LLMs spontaneously organize emotions hierarchically, mirroring human psychology, but also replicate human biases in emotion recognition.
• THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models — Safety in multi-turn LLM interactions is trajectory-dependent, requiring defenses that model temporal risk accumulation.
• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.