Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition

2026-07-09 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Social Sciences & Behavioral Studies · Depth: Expert, medium

Summary

The JAM (Judge for Adaptive Metric-Alignment) framework introduces a theory-agnostic approach to personality recognition, moving beyond models constrained by predefined psychological taxonomies. It aims to discover unified latent pseudo-facets that capture shared psychological structure directly from textual samples. JAM employs an Attention-Pooled Graph Prototypical Network for structured representation learning via clustering in embedding space. It also features a Cross-Theory Harmonization (CTH) approach, integrating Human-Guided Linkage and Machine-Induced Consensus to unify heterogeneous datasets without predefined labels. Furthermore, an LLM-as-a-Judge mechanism, configured as "LLM-before-the-loop" and "LLM-in-the-loop," identifies ambiguous samples to guide adaptive metric learning, improving robustness and data quality. Experiments demonstrate enhanced cross-framework generalization and performance, supporting low-resource personality theories.

Key takeaway

For NLP engineers developing personality recognition systems, this work offers a robust method to overcome limitations of theory-dependent models. By adopting the JAM framework's theory-agnostic approach and LLM-as-a-Judge mechanism, you can achieve better cross-framework generalization and infer latent psychological profiles directly from text, even with limited labeled data. Consider integrating similar adaptive metric learning and cross-theory harmonization for more flexible and accurate systems.

Key insights

A theory-agnostic framework uses LLMs and graph networks to discover unified latent personality traits from text.

Principles

• Personality is better understood as theory-invariant
• Unify heterogeneous datasets without predefined labels
• Adaptive metric learning improves model robustness

Method

JAM learns structured representations via clustering in embedding space using an Attention-Pooled Graph Prototypical Network. It unifies datasets with Cross-Theory Harmonization and uses LLM-as-a-Judge for adaptive metric learning.

In practice

• Infer latent psychological profiles directly from text
• Support low-resource personality theories

Topics

• Personality Recognition
• Large Language Models
• Prototypical Networks
• Metric Learning
• Graph Neural Networks
• Theory-Agnostic AI

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition — JAM enables theory-agnostic personality recognition by learning unified latent psychological structures directly from text, guided by LLMs.
• Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition — JAM enables theory-agnostic personality recognition by learning unified latent "pseudo-facets" across diverse psychological frameworks.
• Hierarchical Multi-Stage Modeling of Adaptive and Maladaptive Self-States in Social Media Timelines — Integrating multi-task learning and structured generation enhances interpretable mental health analysis from social media data.
• Persona Cartography: Charting Language Model Personality Traits in Weight Space — LLM personas can be precisely controlled and measured as positions in a behavioral trait space using the OCEAN framework and low-rank adapters.
• Efficient Safety Alignment of Language Models via Latent Personality Traits — Latent Personality Alignment uses harm-agnostic psychometric statements and adversarial training to robustly align LLMs against jailbreaks with high efficiency.
• Assessing Capabilities of Large Language Models in Social Media Analytics: A Multi-task Quest — Modern LLMs were systematically evaluated across key social media analytics tasks using a Twitter (X) dataset.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.