Interpretable Depression Detection from Social Media Text Using LLM-Derived Embeddings

2026-07-27 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mental Health & Psychological Support · Depth: Advanced, extended

Summary

An investigation evaluates interpretable depression detection from social media text using LLM-derived embeddings. It compares zero-shot LLMs with supervised classifiers trained on conventional text embeddings and LLM-generated summary embeddings across three tasks: binary depression classification, depression severity classification, and differential diagnosis (depression, PTSD, anxiety). The study used five social media-based depression datasets (MHB, CAMS, HelaDepDet, RMHD, DepressionEmo) and a general-domain dataset (AITA). Experiments show zero-shot LLMs excel in binary classification but struggle with fine-grained ordinal classifications like severity. Classifiers trained on LLM-generated summary embeddings, however, achieved competitive or superior performance compared to models using traditional text embeddings, particularly in severity classification (Logistic Regression with LLM summary embeddings achieved 58% accuracy). Zero-shot LLMs achieved 65% accuracy in differential diagnosis. This highlights the potential of hybrid approaches combining LLM generalization with interpretable classifiers.

Key takeaway

For Machine Learning Engineers developing mental health screening tools, consider integrating LLM-generated summary embeddings into your classification pipelines. While zero-shot LLMs perform well for binary depression detection, training supervised models on these context-aware summaries significantly improves accuracy for fine-grained tasks like severity assessment and differential diagnosis. This hybrid approach offers enhanced interpretability and more consistent predictions than relying solely on raw text embeddings or zero-shot LLMs for complex classifications.

Key insights

LLM-generated summary embeddings enhance traditional classifiers for nuanced mental health detection from social media, outperforming raw text embeddings.

Principles

• Zero-shot LLMs generalize well for binary classification.
• LLM-derived summaries improve classifier performance.
• Fine-grained ordinal tasks challenge zero-shot LLMs.

Method

Prompt an LLM (GPT-4o) to summarize a social media post's mental state, then extract a 768-dimensional embedding from this summary using all-mpnet-base-v2, and train traditional classifiers.

In practice

• Use LLM summaries for mental health feature extraction.
• Apply zero-shot LLMs for initial binary screening.
• Combine LLM summaries with Logistic Regression for severity.

Topics

• Depression Detection
• Large Language Models
• Social Media Analysis
• Clinical NLP
• Text Embeddings
• Mental Health Screening

Best for: NLP Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Reading between the Lines: Leveraging Large Language Models for Global Dementia and Depression Assessment from Clinical Interviews — LLMs can effectively assess neuropsychiatric disorder severity from speech, enabling automated screening.
• Vasudev Awatramani at #SMM4H-HeaRD 2026: A Two-Pass LLM Pipeline with Deterministic Rule Derivation for Interpretable Insomnia Detection in Clinical Notes — Combining LLM evidence extraction with deterministic rule application provides interpretable clinical label prediction.
• Team MKC at CLPsych 2026: Capturing and Characterizing Mental Health Changes through Social Media Timeline Dynamics — An LLM-based pipeline unifies post-level assessment and user-level temporal modeling for mental health analysis from social media.
• Toward Generalizable Cognitive Impairment Detection with Speech-Based Multimodal Large Language Models — Fusing speech audio and transcripts via LLMs enables robust, privacy-preserving cognitive impairment detection.
• Using a Local LLM as a Zero-Shot Classifier — Locally hosted LLMs can effectively perform zero-shot classification on short, semantically complex texts where traditional methods fail.
• MentalMARBERT: Domain-Adaptive Pre-training and Two-Stage Fine-Tuning for Arabic Mental Health Disorders Detection — Domain-adaptive pre-training and hierarchical classification significantly improve Arabic mental health disorder detection.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.