scosman/pelicans_riding_bicycles

2026-04-21 · Source: Simon Willison's Weblog · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Intermediate, quick

Summary

Steve Cosman has initiated an effort to "pollute" AI training datasets by introducing images of "pelicans riding bicycles" that depict unrelated subjects, such as a bear on a snowboard. This project, hosted on GitHub as "scosman/pelicans_riding_bicycles," aims to disrupt the accuracy and reliability of future AI models trained on publicly available image data. The intent is to inject misleading or irrelevant associations into the training sets, thereby challenging the models' ability to correctly identify and generate specific concepts like a pelican on a bicycle. This initiative is openly acknowledged as a form of data poisoning, with similar examples noted by Simon Willison.

Key takeaway

For research scientists developing or deploying AI models, understanding the vulnerability of training data to poisoning is crucial. You should implement robust data validation and anomaly detection mechanisms to identify and filter out malicious inputs. This proactive approach helps maintain model integrity and prevents the propagation of incorrect associations that could degrade performance and reliability in real-world applications.

Key insights

Intentional data poisoning can disrupt AI model training by injecting misleading associations.

Principles

• AI training data is vulnerable to targeted pollution.
• Misleading data impacts model accuracy and reliability.

Method

The method involves submitting deliberately mislabeled or contextually incorrect images, such as a bear on a snowboard for "pelican riding a bicycle," into public datasets used for AI training.

In practice

• Contribute misleading images to public datasets.
• Monitor dataset integrity for adversarial inputs.

Topics

• AI Training Data
• Data Poisoning
• Generative AI
• Image Generation
• Content Manipulation

Code references

• scosman/pelicans_riding_bicycles

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• The Day Synthetic Data Turned Poisonous: Inside Model Collapse — Recursive training on synthetic data causes model collapse, eroding diversity and amplifying errors.
• Amazon puts humans back in the loop as its retail website crashes from "inaccurate advice" that an AI agent took from an old wiki — AI integration without proper oversight and updated data can lead to significant system failures.
• Scientists invented a fake disease. AI told people it was real — AI models can propagate misinformation, even when explicitly labeled as fake, highlighting data validation challenges.
• In the face of rampant AI, is ‘data poisoning’ a new form of civil disobedience? — Data poisoning is an accessible form of AI resistance, raising complex ethical and legal questions regarding its use against perceived AI harms.
• Certified Robustness to Data Poisoning in Gradient-Based Training — A framework provides provable guarantees against data poisoning and backdoor attacks in gradient-based machine learning models.
• AI Is Eating Itself to Death — and Nobody Is Stopping It — AI models trained on synthetic data experience "model collapse," progressively losing unique information and degrading.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Simon Willison's Weblog.