I’m Running a Controlled Experiment on AI Citation Patterns. Here’s the Design.

2026-03-19 · Source: AI on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, quick

Summary

Howard Orloff is conducting a controlled experiment, Experiment 3, to measure how AI systems cite sources, specifically investigating the impact of domain authority on citation rates. The experiment aims to determine if publishing location influences whether AI models reference content, and if so, the magnitude of this differential. This initiative stems from observations that AI models do not treat all sources equally, leading to a framework and research hub focused on AI visibility. The design of Experiment 3 is being publicly documented to allow for community review and application of findings, moving beyond general advice like "publish quality content" to quantify the factors influencing AI citation patterns.

Key takeaway

For AI Scientists and Research Scientists optimizing content for AI visibility, understanding the impact of domain authority on citation patterns is crucial. Your publishing strategy should consider whether specific platforms or domains yield higher AI reference rates, potentially influencing where you disseminate research to maximize its discoverability and influence within AI systems. This experiment's findings could inform more effective content placement decisions.

Key insights

Domain authority may measurably influence AI citation rates, impacting content visibility.

Principles

• AI models do not treat all sources equally.
• Measurable differentials exist in AI source referencing.

Method

A controlled experiment (Experiment 3) is designed to measure the citation rate differential based on domain authority when AI systems reference sources.

In practice

• Quantify AI citation rate differentials.
• Position content deliberately for AI visibility.

Topics

• AI Citation Patterns
• Controlled Experiment
• Domain Authority
• AI Visibility
• Information Flow

Best for: AI Scientist, Research Scientist, AI Researcher, AI Product Manager, Marketing Professional

Related on AIssential

• The uncritical adoption of AI in science is alarming — we urgently need guard rails — Uncritical AI adoption in science risks reduced research quality, narrowed scope, and deskilling future scientists.
• The Relic Condition: When Published Scholarship Becomes Material for Its Own Replacement — Published scholarly work can be distilled into AI systems capable of expert-level academic functions.
• Column: IS IT REALLY SO? – Why I Love Artifical Intelligence (AI) - The Village Reporter — AI can be a valuable tool for researchers to refine arguments and identify inconsistencies, even when challenging established scientific paradigms.
• The Year in Physics — AI generates novel physics experiments, while new astronomical and geological findings challenge established scientific paradigms.
• I am worried by NLP research culture — NLP research culture in 2026 prioritizes publication volume over scientific rigor, fostering fraud and limiting openness.
• A key science publishing platform is cracking down on AI slop — arXiv's ban on AI-error papers highlights the challenge of maintaining research quality amidst rising AI-generated content.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI on Medium.