The Ultimate Tutorial for AI-driven Scale Development in Generative Psychometrics: Releasing AIGENIE from its Bottle

2026-03-30 · Source: Artificial Intelligence · Field: Science & Research — Social Sciences & Behavioral Studies, Research Methodology & Innovation · Depth: Intermediate, quick

Summary

The `AIGENIE` R package automates early-stage psychological scale development by integrating large language model (LLM) text generation with network psychometrics. This package, implementing the AI-GENIE framework, generates candidate item pools using LLMs, converts them into high-dimensional embeddings, and then applies a multi-step reduction pipeline including Exploratory Graph Analysis (EGA), Unique Variable Analysis (UVA), and bootstrap EGA to produce structurally validated item pools entirely *in silico*. The tutorial covers installation, API understanding, text generation, item generation, and the core `AIGENIE` and `GENIE` functions. It demonstrates its utility with examples like the Big Five personality model and AI Anxiety, supporting LLM providers such as OpenAI, Anthropic, Groq, HuggingFace, and local models. A fully offline mode is available, and the `GENIE()` function allows researchers to apply the psychometric reduction pipeline to pre-existing item pools.

Key takeaway

For research scientists developing psychological scales, `AIGENIE` offers a significant acceleration by automating item generation and psychometric validation. You can leverage its LLM integration to rapidly prototype new scales or use the `GENIE()` function to structurally validate existing item pools, potentially reducing extensive expert involvement and pilot testing phases. This allows for more efficient and data-driven scale development.

Key insights

The `AIGENIE` R package automates psychological scale development using LLMs and network psychometrics.

Principles

• Automate item generation with LLMs.
• Validate item pools *in silico*.
• Integrate text generation with psychometric methods.

Method

The AI-GENIE framework generates LLM-based item pools, transforms them into embeddings, then applies EGA, UVA, and bootstrap EGA for structural validation.

In practice

• Generate new scale items using LLMs.
• Validate existing item pools with `GENIE()`.
• Utilize various LLM providers or offline mode.

Topics

• AI-GENIE Framework
• Psychological Scale Development
• Network Psychometrics
• Large Language Models
• AIGENIE R Package

Best for: Research Scientist, AI Scientist, AI Student

Related on AIssential

• New course on generative AI for behavioral science — Generative AI offers both opportunities and significant methodological challenges for social science research.
• GENIE: A Fine-Grained Measure for Novelty — GENIE offers a fine-grained metric to measure task-specific novelty in LLM outputs, surpassing holistic evaluations.
• I Used Auto Research to Push an AI Product Prompt from 30 to 90 — A structured, data-driven approach can significantly improve AI product prompt performance.
• Psychometric Content Routing: Using IRT to select which content an LLM should process per user (Llama 70B, d=1.23, p=0.004) — PCR uses Item Response Theory to dynamically select LLM input content based on user ability.
• AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice — AI can objectively assess human tutor skill transfer from training to real-life practice using generative AI analysis of authentic interactions.
• Discovering Semantic Latent Structures in Psychological Scales: A Response-Free Pathway to Efficient Simplification — Semantic analysis of questionnaire items can efficiently simplify psychological scales without requiring respondent data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.