How Human Feedback Shapes AI-generated Community Notes

2026-06-29 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Social Sciences & Behavioral Studies · Depth: Advanced, quick

Summary

A systematic analysis of X's Collaborative Notes system, which integrates LLM-drafted content with human feedback for social media moderation, reveals key dynamics. Researchers examined 19,146 collaborative notes and 211,850 instances of human feedback, finding that suggestions for factual corrections and additional context are most frequently incorporated, while subjective policy judgments are rarely adopted. Human feedback significantly enhances note helpfulness, particularly when active contributors challenge the main claim of a draft. Despite these improvements, collaborative notes achieve "helpful" status and platform visibility at lower rates than human-only or AI-only notes, primarily due to limited human participation. However, collaborative notes serve a complementary function, often addressing posts that human-only or AI-only systems overlook.

Key takeaway

For AI Scientists developing hybrid content moderation systems, you should prioritize mechanisms that facilitate factual corrections and contextual additions, as these are most effective. Focus on engaging active contributors to challenge AI-generated claims, as their feedback significantly improves helpfulness. Consider deploying collaborative notes to address content gaps where human-only or AI-only systems are less active, rather than as direct substitutes, to maximize their complementary value.

Key insights

Human feedback refines AI-drafted content, improving helpfulness but facing adoption bottlenecks due to participation limits.

Principles

• Factual corrections drive AI content improvement.
• Challenging claims from active users boosts helpfulness.
• AI-human collaboration complements existing moderation.

Method

The study systematically analyzed 19,146 collaborative notes and 211,850 human feedback instances to categorize suggestions, track helpfulness changes, and assess platform adoption rates.

In practice

• Prioritize factual accuracy in AI drafts.
• Encourage active users to challenge claims.
• Focus AI-human systems on niche content.

Topics

• Community Notes
• AI Moderation
• Human Feedback
• Large Language Models
• Content Moderation
• Crowdsourcing

Best for: AI Scientist, Research Scientist, AI Ethicist

Related on AIssential

• X tests collaborative AI notes for Community Notes feature — AI-generated drafts, refined by community feedback, can significantly accelerate content moderation processes.
• Toward Human-Centered AI-Assisted Terminology Work — Human-centered AI integrates automation with human oversight to enhance terminology work while mitigating generative AI's risks.
• The (AI) Accusation was Made. Research was Done. — Generative AI challenges human authorship by eroding reader trust and the social recognition essential for legitimacy.
• How to Humanize AI Text: What We Learned from 19,804 Text Pairs — Targeted AI text rewriting can substantially lower detector scores, but its effectiveness varies by topic and measurement.
• AI is teaching us to speak like bots, and it’s a problem — AI chatbots are subtly homogenizing human language, making our communication increasingly machine-like by nudging expression towards an average.
• How people are using generative AI, and what this means for news — Public engagement with generative AI is rapidly increasing, yet trust and comfort with AI in news remain conditional and task-specific.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.