Neural Nexus at PsyDefDetect: Fine-Tuning RoBERTa with Focal Loss and Role-Tagged Dialogue History for Defense Level Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Life Sciences & Biology, Mental Health & Psychological Support · Depth: Expert, quick

Summary

A system developed for the BioNLP 2026 PsyDefDetect shared task classifies help-seeker utterances in multi-turn supportive conversations into nine psychological defense mechanism levels, as defined by the Defense Mechanism Rating Scales (DMRS). The approach fine-tunes roberta-base using a composite training objective that combines focal loss, label smoothing, and square-root dampened class weights to mitigate severe label imbalance in the PSYDEFCONV corpus, where the dominant class accounts for 52% of the training data. Input representation involves concatenating up to eight dialogue turns with role-specific tags, separated by RoBERTa's native "/s" tokens, with the target utterance marked by a "[TARGET]" token. Model selection utilized macro-F1 based early stopping on a stratified 15% validation split and cosine learning rate decay. The best submission achieved an official Leaderboard 1 macro-F1 score of 0.2556, ranking 11th among 21 teams.

Key takeaway

For NLP Engineers or Research Scientists developing dialogue systems that classify nuanced psychological states from conversational data, especially with severe label imbalance, you should consider implementing a composite training objective. Specifically, combine focal loss, label smoothing, and square-root dampened class weights to improve model performance on underrepresented classes. Additionally, integrate role-specific dialogue history and target utterance markers into your input representation to provide crucial contextual cues for better classification accuracy.

Key insights

Fine-tuning RoBERTa with a composite loss and role-tagged dialogue history improves psychological defense level detection in imbalanced conversational data.

Principles

• Addressing severe label imbalance is critical for multi-class classification.
• Contextual dialogue history with role tags enhances utterance classification.
• Composite loss functions can stabilize training for complex tasks.

Method

Fine-tune roberta-base using a composite objective of focal loss, label smoothing, and square-root dampened class weights, with input constructed from role-tagged dialogue turns and a "[TARGET]" token.

In practice

• Apply focal loss and class weights for imbalanced text classification.
• Incorporate role-specific dialogue history for conversational AI tasks.
• Use macro-F1 for early stopping in imbalanced multi-class scenarios.

Topics

• RoBERTa
• Focal Loss
• Dialogue Systems
• Text Classification
• Label Imbalance
• Psychological Defense Mechanisms
• BioNLP 2026

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Titans-as-a-Layer: Test-Time Memory for Conversational Speech Emotion Recognition — Test-time neural memory, via a plug-and-play adapter, significantly improves conversational speech emotion recognition by supplying per-dialogue context.
• Can Machines Tell Whether Gamers Love or Hate a Game? — Fine-tuning Transformer models like RoBERTa significantly improves sentiment analysis accuracy on imbalanced, complex human language datasets.
• Few Shades of Supervision for Discourse Segmentation — Classical fine-tuning of LLMs is most effective for EDU segmentation in spontaneous speech.
• Forget Attention: Importance-Aware Attention Is All You Need — SISA integrates SSM importance directly into attention scores for improved hybrid language model performance.
• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models — Aligning multi-turn LLM behavior with full-context teacher responses mitigates self-anchored drift.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.