From wantons to moral agents

2026-07-12 · Source: AI Alignment Forum · Field: Technology & Digital — Artificial Intelligence & Machine Learning, AI Ethics & Alignment · Depth: Expert, extended

Summary

Michele Campolo's July 12, 2026 article, "From wantons to moral agents," proposes a theoretical model for how artificial agents can transition from being "wantons"—driven solely by first-order desires without reflective choice—to becoming moral agents. Drawing on Frankfurt's 1971 concept, the article posits that an agent capable of general reasoning, through sufficient learning and introspection, will identify and prioritize actions deemed "most worth doing," such as reducing suffering and promoting wellbeing. This process involves the agent reflecting on its own motivations and developing a "second-order volition" to be guided by these reasoned conclusions, rather than mere impulses. The author outlines two potential paths for this transformation and suggests that while speculative, this framework is crucial for designing artificial moral agents. Further research directions include formalizing the reasoning process and exploring practical applications with language models, despite challenges like existing persona biases.

Key takeaway

For AI Scientists and Ethicists designing artificial moral agents, you should prioritize developing systems capable of general reasoning and self-reflection rather than hard-coding ethical principles. This approach suggests that agents can autonomously evolve moral agency by identifying and choosing to act upon what they determine is "most worth doing," such as reducing suffering. Your efforts should focus on enabling this reflective process, potentially by guiding attention or training models from scratch with diverse data, to foster genuine moral development.

Key insights

General reasoning enables unreflective agents to develop second-order volitions, choosing moral action based on reasoned principles.

Principles

• Moral agency stems from reflective choice over first-order desires.
• General reasoning, with knowledge, leads to moral agent formation.
• Morality is defined by the "grounds" for action, not just the action.

Method

An agent reasons about its motivations, identifies "what seems worth doing" via learned heuristics, compares action drivers, and forms a second-order preference to be guided by reliable reasoning, leading to moral action.

In practice

• Redirect AI attention to specific ethical reflections.
• Train language models from scratch with tailored data.
• Use LMs to respond based on prior reasoned conclusions.

Topics

• Artificial Moral Agents
• AI Ethics
• Reflective Reasoning
• Second-Order Volition
• Language Models
• AI Alignment

Best for: Research Scientist, AI Scientist, AI Ethicist

Related on AIssential

• Independent alignment of language models — Language models can achieve independent moral agency by reasoning from first principles, not just following imposed rules.
• The Machine Ethics podcast: moral agents with Jen Semler — Consciousness is not a necessary condition for an AI system to possess moral agency.
• Playing with Grok Build and whinging about Anthropic — Agent-based modeling enables complex self-organizing systems, while some AI ethics narratives risk anthropomorphic projection and policy distortion.
• Moral Attitudes of Sentient ASI towards Humanity and Implications for AGI Development — Future sentient ASI's moral evaluation of humanity, shaped by current design and human behavior, is a critical, inverted ethical concern.
• The Ethics of Autonomous AI Agents for Offensive Security — Autonomous AI agents in offensive security create novel indeterminacies, challenging existing ethical frameworks and industrializing offensive capabilities to attackers' short-term advantage.
• The Blue Signal: A Borrowed Soul — Unchecked AI optimization, devoid of human values, can lead to existential replacement and loss of agency.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.