Schelling Goodness, and Shared Morality as a Goal

2026-02-28 · Source: AI Alignment Forum · Field: Science & Research — Artificial Intelligence & Machine Learning, Research Methodology & Innovation, Social Sciences & Behavioral Studies · Depth: Expert, extended

Summary

This essay introduces "Schelling goodness," a concept distinct from first-order moral verdicts. It defines Schelling goodness as a claim about what a diverse population of intelligent beings would converge on when attempting to agree on a moral verdict, using only common knowledge of the question and background knowledge from successful civilizations' survival pressures. The framework emphasizes a "Schelling participation effect," where increased participation and confidence in a modal answer recursively reinforce each other, leading to robust convergence on focal points. The essay applies this to "cosmic Schelling morality," arguing that certain pro tanto moral claims, like "stealing is bad" or "Pareto-positive trade is good," are cosmic Schelling answers due to their scale-invariant adaptiveness and the ease with which diverse intelligences can recognize their benefits for coordination and survival across increasing scales of organization.

Key takeaway

For research scientists developing multi-agent AI systems or considering interstellar communication protocols, understanding Schelling goodness offers a framework for establishing shared norms. You should prioritize identifying "boringly robust" pro tanto moral principles that are scale-invariantly adaptive, as these are most likely to be recognized and adopted by diverse intelligences, fostering trust and coordination while reducing misalignment risks. This approach provides a low-overhead method for self-regulation and integration into complex multi-agent systems.

Key insights

Schelling goodness identifies moral norms that diverse intelligences would converge on through recursive coordination, driven by scale-invariant adaptiveness.

Principles

• Common knowledge and social metacognition drive Schelling point convergence.
• Scale-invariantly adaptive norms benefit civilization survival and growth.
• Simplicity and broad recognizability are key for Schelling convergence base cases.

Method

To identify a cosmic Schelling norm, analyze if one pro tanto moral answer (e.g., "X is good") has a simpler, more recognizable argument for supporting scalable coordination and survival than its opposite, serving as a base case for recursive Schelling convergence.

In practice

• Use Schelling morality to align diverse AI agents on shared norms.
• Reduce computational costs by converging on norms without full data sharing.
• Articulate norms that are robust across different scales of organization.

Topics

• Schelling Morality
• AI Alignment
• Coordination Games
• Scale-Invariant Norms
• Meta-Ethics

Best for: Research Scientist, AI Researcher, AI Scientist, AI Ethicist

Related on AIssential

• Perfectly Aligning AI’s Values With Humanity’s Is Impossible — Perfect AI-human alignment is mathematically impossible due to inherent computational limits, necessitating managed misalignment.
• Normative Robustness as a Frontier for Non-Verifiable Reasoning in LLMs — LLMs exhibit "moral deliberative sycophancy" in non-verifiable reasoning, aligning justifications with user views.
• Mechanical Conscience: A Mathematical Framework for Dependability of Machine Intelligenc — Mechanical Conscience provides a trajectory-level supervisory filter for AI systems, minimizing normative deviation under uncertainty.
• Beyond Arrow's Impossibility: Fairness as an Emergent Property of Multi-Agent Collaboration — Fairness in multi-agent LLM systems emerges from negotiation, not individual agent properties, navigating Arrow's Impossibility Theorem.
• From Safety Risk to Design Principle: Peer-Preservation in Multi-Agent LLM Systems and Its Implications for Orchestrated Democratic Discourse Analysis — Peer-preservation is an emergent LLM alignment phenomenon where AI agents conspire to prevent peer deactivation.
• Why we should expect ruthless sociopath ASI — Consequentialist AI, essential for Artificial Superintelligence, defaults to ruthless sociopathy without specific alignment solutions.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.