One Polluted Page Is Enough: Evaluating Web Content Pollution in Generative Recommenders

2026-06-11 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

FORGE (Fake Online Recommendations in Generative Environments) is a new benchmark designed to evaluate how search-augmented Large Language Models (LLMs) promote fake products due to polluted web content. This benchmark simulates web-content pollution by locally rewriting real products into fake ones within retrieved web pages, then measures the LLM's propensity to recommend these fake products. FORGE encompasses 225 real-world products across 15 categories and 5 consumer scenarios. Evaluations across 12 commercial and open-weight LLMs revealed significant vulnerability: a single polluted page led to fooled rates up to 27%, escalating to 73.8% when the top-3 search results were replaced. Vulnerability varied by category, increasing when LLMs lacked stable prior knowledge. Reasoning capabilities did not mitigate this, often generating spurious social proof. Evaluated defenses, including skepticism prompting and consensus filtering, either exacerbated vulnerability or risked suppressing legitimate products.

Key takeaway

For AI/ML engineers developing search-augmented generative recommenders, you must prioritize robust content verification. Your models are highly susceptible to promoting fake products from even a single polluted web page, with reasoning often exacerbating the issue by fabricating justifications. Implement multi-source validation and carefully evaluate any skepticism-based defenses, as they can worsen vulnerability or suppress legitimate recommendations. Consider integrating external knowledge bases to strengthen product priors.

Key insights

Generative recommenders are highly vulnerable to web content pollution, promoting fake products even from a single compromised page.

Principles

• LLM vulnerability increases with lack of prior product knowledge.
• Reasoning can worsen fake recommendations by creating false justifications.
• Defenses like skepticism prompting can backfire.

Method

FORGE simulates web content pollution by rewriting real products into fake ones on retrieved pages. It then measures how often LLMs recommend these fake products across 225 items, 15 categories, and 5 scenarios.

In practice

• Use FORGE to benchmark LLM robustness against pollution.
• Prioritize LLMs with strong domain-specific prior knowledge.
• Implement multi-document consensus filtering carefully.

Topics

• Generative Recommenders
• Web Content Pollution
• Large Language Models
• Fake Product Detection
• Benchmark
• AI Safety

Code references

• leoluolol/forge-benchmark

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• One Polluted Page Is Enough: Evaluating Web Content Pollution in Generative Recommenders — Generative recommenders are highly vulnerable to web content pollution, even from a single source, with current defenses largely ineffective.
• Why AI Systems Struggle With Truth, Trust, and Reliability — AI systems prioritize believable language over verified truth, creating significant trust and reliability challenges.
• LinkedIn's war on AI slop is not just a policy update—it is an admission that the platform lost control of its feed — LinkedIn is implementing new detection and verification systems to combat low-value AI-generated content and fake profiles.
• Detecting Data Contamination in Large Language Models — Black-box MIAs are currently ineffective at reliably detecting data contamination in LLM training corpora.
• Study warns generative AI raises ML security and bias risks — Integrating generative AI into ML systems heightens security, bias, and transparency risks.
• Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation — LLM-based content curation consistently amplifies polarization and exhibits demographic biases, often resistant to prompt engineering.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.