New course on generative AI for behavioral science

2026-03-10 · Source: Statistical Modeling, Causal Inference, and Social Science · Field: Science & Research — Social Sciences & Behavioral Studies, Research Methodology & Innovation, Artificial Intelligence & Machine Learning · Depth: Advanced, long

Summary

Northwestern University offered a graduate seminar titled "Generative AI for Social Science" during the winter quarter, co-taught by Jessica and Aaron Shaw. The course surveyed emerging applications of generative AI, primarily large language model (LLM) agents, within social sciences, focusing on methodological and metascientific concerns when AI simulates or substitutes human observations. The seminar combined computer science and communications students, fostering diverse discussions on survey methods, psychology, and transformer mechanisms. Key topics included LLMs as surrogates for human attitudes, opinions, cognition, and behavior, as well as the biases and threats to generalization posed by LLM simulations. The curriculum also covered validation methods, including heuristic and statistical approaches, and explored AI's role in causal discovery, explanation, and generating research ideas, culminating in discussions on belief-like representations and Bayesian inference in LLMs.

Key takeaway

For AI Scientists developing or applying LLMs in social science research, it is critical to rigorously validate models against human data. You should prioritize understanding potential biases and non-human-like errors in LLM simulations to ensure valid inferences about human behavior. Embrace interdisciplinary collaboration and statistical validation frameworks to enhance the reliability and generalizability of your AI-driven social science findings.

Key insights

Generative AI offers both opportunities and significant methodological challenges for social science research.

Principles

• LLMs can simulate human behavior.
• Validation is crucial for LLM-based social simulations.
• AI can aid in research design and causal inference.

Method

The course structure involved interdisciplinary collaboration between CS and Communications students, integrating discussions on LLM internals with social science methodologies, and included a workshop on validating generative AI for social science.

In practice

• Explore new prompting architectures based on cognitive theory.
• Apply ML interpretability to steer models with social science insights.
• Study belief elicitation and uncertainty expression in LLMs.

Topics

• Generative AI
• Social Science Applications
• Large Language Models
• AI Simulation Validation
• Causal Discovery

Best for: AI Scientist, AI Student, AI Researcher, Research Scientist

Related on AIssential

• Generative AI Meets Data Engineering: What Happens When You Can Describe Your Database in Plain… — LLMs can generate database schemas from natural language, but require robust validation and human oversight for production reliability.
• Fake users generated by AI can't simulate humans — review of 182 research papers — LLMs fail to simulate human cognition and behavior due to their training on text, not diverse lived experience.
• Beyond Static Responses: Multi-Agent LLM Systems as a New Paradigm for Social Science Research — A six-tier framework maps LLM agent evolution from static tools to complex adaptive systems for social science research.
• Will AI ruin the social sciences — or revolutionize them? — Large language models are simultaneously compromising social science research data integrity and offering tools for enhanced analytical robustness.
• Evaluating LLMs as Human Surrogates in Controlled Experiments — LLMs can reproduce directional behavioral effects but often miscalibrate effect magnitudes compared to human data.
• When Do LLMs Generate Realistic Social Networks? A Multi-Dimensional Study of Culture, Language, Scale, and Method — LLM-generated social networks are significantly influenced by prompt design, culture, language, and model scale.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Statistical Modeling, Causal Inference, and Social Science.