Facet-Informed Prompting for LLM-Based Personality Assessment: Error-Guided Exemplar Selection and Hierarchical Prediction

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Science & Research — Social Sciences & Behavioral Studies, Health & Medical Research, Research Methodology & Innovation · Depth: Expert, medium

Summary

The paper "Facet-Informed Prompting for LLM-Based Personality Assessment: Error-Guided Exemplar Selection and Hierarchical Prediction" by Hussain, Shah, Oltmanns, and Gupta, presented at CLPsych 2026, introduces a structured prompting approach for LLM-based personality assessment. This method addresses limitations of prior work, which often uses coarse binary labels and direct domain-level predictions, by incorporating fine-grained facet-level predictions and domain-level predictions informed by facet outputs. The approach utilizes a five-level ordinal label scheme for predictions, ranging from "very low" to "very high" trait expression. A key component is an error-guided self-refinement procedure using in-context learning (ICL) to enhance accuracy. The study evaluates both zero-shot and one-shot prompts, with one-shot prompts using a single demonstration example selected via the refinement process. This framework aims to provide a systematic, interpretable, and principled method for assessing personality from long-form narratives by combining hierarchical domain-facet relationships with structured prompting and refinement.

Key takeaway

For AI Scientists and NLP Engineers developing LLM-based personality assessment tools, consider implementing a structured prompting approach that integrates hierarchical facet-level predictions. This method, combined with error-guided in-context learning and a five-level ordinal label scheme, can significantly enhance interpretability and prediction accuracy. You should explore incorporating fine-grained facet outputs to inform domain-level predictions, moving beyond coarse binary labels for more nuanced and principled assessments from long-form narratives.

Key insights

Structured prompting with error-guided refinement and hierarchical facet prediction improves LLM-based personality assessment interpretability and accuracy.

Principles

• Personality assessment benefits from hierarchical facet structure.
• Error-guided self-refinement enhances LLM prediction accuracy.
• Ordinal label schemes capture trait expression continuum.

Method

A structured prompting approach with three objectives: direct domain, fine-grained facet, and facet-informed domain prediction. It uses a five-level ordinal scheme and error-guided ICL for self-refinement, evaluating zero-shot and one-shot prompts.

In practice

• Apply error-guided ICL for LLM refinement.
• Use ordinal labels for nuanced trait expression.
• Structure prompts for hierarchical personality assessment.

Topics

• LLM Personality Assessment
• Structured Prompting
• In-Context Learning
• Error-Guided Refinement
• Hierarchical Prediction
• Computational Linguistics

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost? — Fine-tuning LLMs improves personality questionnaire response stability but fails to induce accurate, multi-dimensional Big Five profiles from unguided text.
• Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring — Adaptive prompt routing with subject-aware LLM evaluation improves tutoring efficiency and student engagement in diverse subjects.
• Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior — LLM self-report-behavior coherence is selective, depending on the psychometric framework (TPB > Big 5) and conversational context.
• LLM Predictive Scoring and Validation: Inferring Experience Ratings from Unstructured Text — LLMs can directionally predict experience ratings from unstructured text, revealing distinct evaluative constructs.
• Dimension-Level Intent Fidelity Evaluation for Large Language Models: Evidence from Structured Prompt Ablation — Dimension-level intent fidelity evaluation uncovers LLM quality deficits missed by holistic assessment.
• LLMs Can Better Capture Human Judgments--With the Right Prompts — Asking better questions to LLMs through specific prompting strategies improves their ability to capture human judgments and response distributions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.