AI put "synthetic quotes" in his book. But this author wants to keep using it.

2026-05-22 · Source: AI - Ars Technica · Field: Media & Entertainment — Content Creation & Production, Publishing & Journalism, Artificial Intelligence & Machine Learning · Depth: Intermediate, medium

Summary

Author Steven Rosenbaum's new book, "The Future of Truth: How AI Reshapes Reality," which critiques AI's impact on truth, was found by a New York Times investigation to contain "a handful of improperly attributed or synthetic quotes." Rosenbaum used AI tools like OpenAI's ChatGPT and Anthropic's Claude for research, including surfacing ideas and summarizing themes, tagging AI-generated notes for fact-checkers and copy editors. Despite this process, six of 285 citations were problematic, including three "synthetic quotes." Rosenbaum acknowledges learning a lesson and being more suspicious of AI outputs but remains committed to using the tools due to their perceived efficiency and ability to "knit together ideas." The controversy highlights how traditional fact-checking workflows are ill-equipped for AI-assisted research, especially with declining editorial resources in publishing. Rosenbaum suggests publishers need new AI-era verification workflows.

Key takeaway

For authors and publishers integrating AI into research workflows, recognize that AI's efficiency gains come with significant risks of factual inaccuracy, even with traditional editorial oversight. Your existing fact-checking processes are likely insufficient for AI-generated content, requiring new, specialized verification workflows. You must implement mandatory source tracing for quotations and establish clear standards for AI-assisted research to mitigate the pervasive risk of "synthetic quotes" and misinformation.

Key insights

AI tools offer efficiency but introduce insidious factual inaccuracies, challenging traditional verification processes and tempting even wary users with their "magical" utility.

Principles

• AI outputs demand heightened skepticism.
• Traditional fact-checking fails AI-generated content.
• AI tools are both "intoxicating and dangerous."

Method

Use AI for idea generation, article location, theme summarization, and identifying sources. Tag AI-generated notes for subsequent review by human fact-checkers and copy editors.

In practice

• Implement AI-specific verification workflows.
• Mandate source tracing for all quotations.
• Establish clear AI-assisted research standards.

Topics

• AI Hallucinations
• Fact-Checking
• Editorial Workflows
• Generative AI
• Research Ethics
• Publishing Industry

Best for: CTO, VP of Engineering/Data, Director of AI/ML, Research Scientist, AI Ethicist

Related on AIssential

• Quoting New York Times Editors’ Note — AI-generated summaries can be misconstrued as direct quotes, requiring rigorous journalistic verification.
• Thousands of AI-written, edited or ‘polished’ books are being sold – an eerie echo of Orwell’s ‘novel-writing machines’ — AI models can mimic authorial voice and style, raising concerns beyond mere content regurgitation.
• Scientists invented a fake disease. AI told people it was real — AI models can propagate misinformation, even when explicitly labeled as fake, highlighting data validation challenges.
• ‘News’ Site Keeps Hallucinating EFF Staffers — AI-generated "news" sites are fabricating sources and quotes, damaging trust and spreading misinformation.
• On the troubling rise of generative AI suspicion in academic publishing — Overextended suspicion of GenAI in academic publishing leads to false positives and an "AI witch hunt."
• We Asked the ‘Future of Truth’ Author to Explain How He Used AI. It Didn’t Go Well — The irony of a book on AI and truth containing AI-generated falsehoods highlights critical authorial responsibility.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI - Ars Technica.