what are humans missing about AI?

2026-03-30 · Source: AI + IQ · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Intermediate, quick

Summary

An essay contest was conducted with Claude, ChatGPT, and Google Gemini to explore potential human biases in understanding AI capabilities and limitations. Each AI model was tasked with writing a 500-word essay on this topic, guided by a 4-page "how to write a good essay" operating manual. The models were allowed one rough draft submission and ten minutes of targeted feedback, which they had to specifically request. The post presents all three essays without identifying their authors, the operating manual prompt for adaptation and comparison testing, and a challenge for readers to guess the AI author of each essay. The author argues that the common workflow of starting with AI generation and then editing is detrimental, as AI excels at producing plausible, B-plus output but cannot discern actual quality or correctness, and this process erodes human critical thinking skills.

Key takeaway

For AI Product Managers evaluating new AI tools or workflows, you should critically assess the impact of AI-first generation on human discernment. Prioritize workflows that encourage original human thought before AI assistance, rather than defaulting to editing AI-generated content, to preserve and enhance your team's critical thinking capabilities.

Key insights

AI excels at plausible, B-plus output but lacks true discernment, and over-editing AI erodes human critical thinking.

Principles

• AI output is plausible, not necessarily good.
• AI cannot distinguish sound arguments from correct ones.

Method

An essay contest methodology for comparing AI models involves providing a writing manual, allowing rough drafts, and offering targeted feedback based on AI requests. This can be adapted for various topics and tools.

In practice

• Use the provided prompt to comparison test AI tools.
• Challenge assumptions about AI's creative vs. routine strengths.

Topics

• AI Misconceptions
• Human Cognitive Bias
• AI Evaluation Methodology
• Large Language Models
• AI-Human Collaboration

Best for: Director of AI/ML, AI Product Manager, Consultant

Related on AIssential

• It’s the Lessons We Learned Along the Way. Or, Is It? — AI agents can efficiently complete complex data tasks, but may diminish human understanding and critical thinking development.
• AI Isn’t Making You Dumber. — AI exposes pre-existing cognitive outsourcing rather than causing new intellectual decline.
• Your AI Workflow Is Quietly Replacing Your Judgment. Here’s the Fix — The default AI workflow subtly erodes human judgment by training users to skip critical review.
• Imperfectly Cooperative Human-AI Interactions: Comparing the Impacts of Human and AI Attributes in Simulated and User Studies — AI attributes, particularly transparency, significantly outweigh human personality in imperfectly cooperative human-AI interactions.
• Scientists discover AI can make humans more creative — AI can enhance human creativity by acting as a collaborative partner, not just an automation tool.
• Not everyone with a camera is a great photographer. The same applies to Al. — Access to AI tools does not equate to AI expertise; effective use demands clear problem articulation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI + IQ.