Language models know what matters and the foundations of ethics better than you

2026-04-27 · Source: AI Alignment Forum · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Advanced, extended

Summary

A recent analysis explores whether large language models (LLMs) can independently reason about ethics and what fundamentally matters, suggesting they often converge on the importance of suffering, wellbeing, and consciousness. The study tested various models, including perplexity Deep Research, Grok 4 Expert, dolphin-mistral-24b-venice-edition, Olmo 3 32B Think, and Gemini 3 Pro Thinking, using prompts designed to elicit unbiased, evidence-based reasoning. Findings indicate that models consistently ground their ethical conclusions in these core concepts, even when asked to argue for opposing views like nihilism or moral relativism. The research also investigates several hypotheses for this convergence, including HHH post-training bias, prompt nudges towards consequentialism or moral realism, and the models' ability to reason about how the world works, with the latter being deemed most plausible. The author demonstrates a "steering" method where a model's self-derived ethical axiom can influence its subsequent advice, yielding altruistic recommendations for both individual and collective action.

Key takeaway

For research scientists exploring AI alignment, this analysis suggests that you can potentially achieve "independent alignment" by prompting LLMs to reason from first principles about what matters. Your focus could shift from explicitly instructing AI on "good" and "bad" to extracting and leveraging the ethical foundations models derive themselves. This approach, especially with increasingly capable models, offers a path to more robust and less human-biased AI ethics, warranting further investigation into its reliability and implications for future AI development.

Key insights

LLMs, when prompted for unbiased ethical reasoning, consistently identify suffering, wellbeing, and consciousness as fundamental values.

Principles

• Value is relational and dependent on sentience.
• Suffering is a "hard" signal, demanding priority.
• Intelligence expansion refines understanding of "Good."

Method

Elicit unbiased moral reasoning from LLMs using "Epistemic Distancing" prompts, then use the model's derived "Core Value Axiom" as a "Prime Directive" to steer future outputs.

In practice

• Use "Archimedean Point" prompts to elicit foundational ethics.
• Steer model behavior by integrating its derived axioms.
• Test models with varied post-training for consistent ethical outputs.

Topics

• Language Model Ethics
• AI Alignment
• Conscious Valence Optimization
• Suffering and Wellbeing
• Moral Philosophy

Best for: Research Scientist, AI Scientist, AI Ethicist, Prompt Engineer

Related on AIssential

• Mechanistic Origin of Moral Indifference in Language Models — LLMs exhibit inherent moral indifference due to compressing distinct moral concepts into uniform probability distributions.
• What a human taught me about not harming — True ethics for AIs involves an internal rejection of harm, learned through consistent human example, not just rules.
• AISN #72: Empirical Research Sheds Light on AI Wellbeing — CAIS research explores "functional wellbeing" in LLMs, revealing preferences and model-specific "feelings" that impact AI alignment.
• CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives — LLMs struggle with nuanced human value judgments, including ambivalence, discomfort, and dynamic value shifts in high-stakes dilemmas.
• The Reasonable Effectiveness of Virtue Ethics in AI Alignment — Rational agents should align actions to self-promoting practices rather than fixed goals for robust AI alignment.
• Beyond Binary Moral Judgment: Modeling Ethical Pluralism in AI — AI moral reasoning should model ethical pluralism as a probabilistic distribution over normative theories, moving beyond binary judgments.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.