RSPC: A Benchmark for Modeling Stress and Psychiatric Conditions in Digitally Mediated Relationships using Psychiatrist Annotations

2026-06-25 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computational Psychiatry · Depth: Expert, quick

Summary

The Relational Stress and Psychiatry Corpus (RSPC) is a new benchmark dataset designed to model mental health conditions within interpersonal contexts, specifically digitally mediated relationships. Comprising 1,799 Reddit posts about long-distance relationships, RSPC features annotations by psychiatrists for diagnostic categories like anxiety and depression, relational stressor triggers, and relationship phases. The benchmark evaluates seven fine-tuned transformer models and five large language models across multi-label disorder classification, relational trigger detection, and temporal phase prediction. Results show task-dependent performance differences, with Claude-3-Haiku achieving the best disorder classification (Macro-F1 = 0.538) and GPT-4o excelling in relational trigger detection (Macro-F1 = 0.519). The study also identifies a strong association between anxiety disorders and chronic relational uncertainty, advocating for context-aware mental health modeling in NLP.

Key takeaway

For NLP Engineers developing mental health applications, this research suggests shifting from isolated condition modeling to context-aware approaches. You should consider integrating relational dynamics and temporal phases into your datasets and models, as demonstrated by the RSPC benchmark. Evaluate specialized LLMs like Claude-3-Haiku for disorder classification and GPT-4o for trigger detection to optimize performance. This can lead to more nuanced and accurate mental health support systems.

Key insights

Modeling mental health in NLP benefits from relational context, moving beyond individual-centric approaches.

Principles

• Interpersonal context is crucial for mental health modeling.
• Different LLMs excel at distinct mental health NLP tasks.
• Anxiety disorders link strongly to relational uncertainty.

Method

Psychiatrists annotated 1,799 Reddit posts for diagnostic categories, relational triggers, and relationship phases, then benchmarked transformer and LLM performance on classification and detection tasks.

In practice

• Use RSPC to develop context-aware mental health NLP models.
• Evaluate LLMs for specific mental health classification tasks.
• Investigate relational uncertainty as an anxiety predictor.

Topics

• Mental Health NLP
• Relational Stress
• Psychiatric Conditions
• LLM Benchmarking
• Psychiatrist Annotation
• Long-Distance Relationships

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Psychological Digital Twins : Conceptual Models, AI Foundations and Ethical Challenges — Robust Psychological Digital Twins require hybrid AI, combining LLM flexibility with symbolic guardrails for precision mental health.
• When PCOS Meets Eating Disorders: An Explainable AI Approach to Detecting the Hidden Triple Burden — Explainable AI can detect co-occurring health conditions like PCOS and eating disorders from social media data.
• Reading between the Lines: Leveraging Large Language Models for Global Dementia and Depression Assessment from Clinical Interviews — LLMs can effectively assess neuropsychiatric disorder severity from speech, enabling automated screening.
• Rethinking Patient Education as Multi-turn Multi-modal Interaction — MedImageEdu benchmarks multi-turn, multimodal patient education, revealing gaps in visual grounding, safety, and handling emotional interactions.
• TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots — TherapyGym evaluates and aligns therapy chatbots using clinical fidelity and safety metrics, improving performance through RL with expert-validated feedback.
• From 'What' to 'How' and 'Why': Sharing LLM-Generated Retrospective Summaries of Older Adults' Passive Tracking Data with Remote Family Members — LLM-generated summaries for remote family members require context-aware narratives beyond raw data to be helpful and trusted.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.