Exploring a New Competency Modeling Process with Large Language Models

2026-02-13 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, AI Applications in Human Resources · Depth: Advanced, quick

Summary

A new competency modeling process, built on large language models (LLMs), addresses the limitations of traditional expert-driven approaches in human resource management. This study reconstructs the competency modeling workflow by decomposing expert practices into structured computational components. LLMs are used to extract behavioral and psychological descriptions from raw textual data and map them to predefined competency libraries via embedding-based similarity. The framework introduces a learnable parameter to adaptively integrate information sources, determining the relative importance of behavioral and psychological signals. An offline evaluation procedure enables systematic model selection without extensive data collection. Empirical results from a software outsourcing company show strong predictive validity, cross-library consistency, and structural robustness, transforming competency modeling into a transparent, data-driven, and evaluable analytical process.

Key takeaway

For HR professionals and talent managers seeking to modernize competency modeling, this LLM-driven framework offers a path to greater objectivity and reproducibility. You can move beyond manual analysis by leveraging AI to extract and map behavioral data, ensuring more consistent and data-driven talent decisions. Consider adopting this approach to enhance the validity and scalability of your talent selection and development programs.

Key insights

LLMs can transform expert-dependent competency modeling into a data-driven, evaluable analytical process.

Principles

• Decompose expert practices into computational components.
• Integrate diverse information sources adaptively.

Method

The method extracts behavioral/psychological descriptions using LLMs, maps them to competency libraries via embedding similarity, and adaptively integrates information sources with a learnable parameter.

In practice

• Use LLMs for behavioral description extraction.
• Map descriptions to libraries using embeddings.
• Implement offline validation for model selection.

Topics

• Large Language Models
• Competency Modeling
• Human Resource Management
• Embedding Similarity

Best for: Executive, AI Scientist, Research Scientist, Machine Learning Engineer, Data Scientist, HR Professional

Related on AIssential

• Assessing the Business Process Modeling Competences of Large Language Models — LLMs show strong syntactic and pragmatic BPMN generation but struggle with semantic accuracy and XML validity.
• Characterising LLM-Generated Competency Questions: a Cross-Domain Empirical Study using Open and Closed Models — CompCQ framework systematically characterizes LLM-generated Competency Questions across multiple linguistic, structural, and semantic dimensions.
• An NLP-Driven Framework for Curriculum-Labor Market Alignment: Schema-Constrained LLM Extraction, ESCO-Anchored Semantic Matching, and Multi-Dimensional Gap Quantification — An NLP framework uses LLMs and semantic matching to quantify curriculum-labor market competency gaps with high reliability.
• Proposing Topic Models and Evaluation Frameworks for Analyzing Associations with External Outcomes: An Application to Leadership Analysis Using Large-Scale Corporate Review Data — Leveraging LLMs for topic modeling improves interpretability, specificity, and sentiment consistency for external outcome analysis.
• Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity — LLMs, guided by a context-aware "Complexity Score" for prompt selection, significantly outperform fine-tuned models on inferentially complex, low-prevalence data extraction.
• Automated reproducibility assessments in the social and behavioral sciences using large language models — Large language models can automate scientific reproducibility checks, matching or exceeding human reanalysis performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.