Self-State Identification with Retrieved In-Context Examples and Open-Weight LLMs

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Mental Health & Psychological Support · Depth: Expert, quick

Summary

A system for the CLPsych 2026 shared task on post-level identification of adaptive and maladaptive self-states achieved competitive results. This system employs a retrieval-augmented in-context learning ensemble, combining two open-weight LLMs: Qwen3.5-27B and Mistral-Small-3.2-24B-Instruct. It utilizes a three-call prompt decomposition strategy, focusing on unified, adaptive-focused, and Affect-focused extraction. Outputs are merged using deterministic aggregation with subtask-tuned element-selection strategies. The system secured 2nd place out of 17 on Task 1.1 (subelement Macro F1 = 0.441) and 5th place out of 17 on Task 1.2 (Avg RMSE = 0.994), demonstrating its effectiveness in subelement classification and presence rating.

Key takeaway

For NLP engineers developing mental health support systems, this work demonstrates that combining open-weight LLMs like Qwen3.5-27B and Mistral-Small-3.2-24B-Instruct with retrieval-augmented in-context learning and prompt decomposition can yield highly competitive results. You should consider this ensemble and decomposition architecture to improve accuracy in complex psychological state identification tasks, especially for subelement classification and presence rating.

Key insights

Retrieval-augmented in-context learning with open-weight LLM ensembles effectively identifies self-states in psychological text.

Principles

• Ensemble open-weight LLMs for robust classification.
• Decompose complex tasks into focused prompt calls.

Method

The system uses a retrieval-augmented in-context learning ensemble of Qwen3.5-27B and Mistral-Small-3.2-24B-Instruct, with a three-call prompt decomposition and deterministic aggregation.

In practice

• Combine Qwen3.5-27B and Mistral-Small-3.2-24B-Instruct.
• Implement three-call prompt decomposition for nuanced extraction.

Topics

• Self-State Identification
• CLPsych Shared Task
• Retrieval-Augmented Generation
• In-Context Learning
• Open-Weight LLMs
• Prompt Engineering
• Ensemble Methods

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• The Architecture Behind Open-Source LLMs — MoE architectures and diverse training strategies drive rapid progress in open-weight LLMs, optimizing cost and performance.
• Lossless Prompt Compression via Dictionary-Encoding and In-Context Learning: Enabling Cost-Effective LLM Analysis of Repetitive Data — LLMs can learn dictionary encodings in-context, enabling lossless prompt compression without fine-tuning and preserving analytical accuracy.
• Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence — Self-function vectors enable robust aleatoric and epistemic uncertainty decomposition in LLM In-Context Learning.
• Generative AI-Based Virtual Assistant using Retrieval-Augmented Generation: An evaluation study for bachelor projects — RAG-based virtual assistants with self-reflection can effectively address LLM limitations in specialized academic contexts.
• Label-Aware Pseudo-Training Sample Generation for Text Classification — A new data augmentation method uses LLMs to subtly alter sentences, improving NLP performance with limited data.
• Context Memorization for Efficient Long Context Generation — Attention-state memory externalizes LLM prefixes into a lookup-based memory, improving long-context inference efficiency and accuracy.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.