The Impact of AI Usage and Informativeness on Skill Development in Logical Reasoning

2026-05-23 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Human-AI Interaction, Educational Psychology & Learning Sciences · Depth: Expert, extended

Summary

A study presented at HHAI 2026, Brussels, Belgium, on July 8–10, 2026, investigated how AI usage and informativeness affect skill development in logical reasoning. Researchers conducted a controlled user experiment with 132 participants solving time-constrained logic puzzles, providing on-demand AI assistance with 100% accuracy but varying informativeness (low-information AI revealed one object, high-information AI revealed three). The findings indicate that greater AI usage correlates with weaker skill development; heavy AI users underperformed comparable peers in post-AI assessments, while light users performed similarly to non-AI users. This effect is moderated by AI informativeness: low-information AI neither improved immediate performance nor preserved post-AI skills, leading to weaker overall learning. Conversely, high-information AI improved short-run performance without reducing post-AI outcomes on average, though effects were heterogeneous. High-ability participants used high-information AI selectively, showing stronger skill growth, whereas lower-ability participants relied on it more, exhibiting weaker learning and inflated confidence.

Key takeaway

For AI Scientists and educators designing learning systems, you must carefully consider AI's informativeness and user heterogeneity. Low-information AI can displace independent effort without performance gains, hindering skill development. High-information AI, while boosting immediate performance, can widen ability gaps, especially for lower-ability users who over-rely. Your designs should prioritize scaffolding and incentives that encourage sustained cognitive engagement, ensuring AI complements rather than substitutes human reasoning.

Key insights

AI's impact on skill development hinges on usage patterns and the informativeness of the assistance provided.

Principles

• Heavy AI reliance hinders skill development.
• Low-information AI can be distracting.
• High-information AI can widen ability gaps.

Method

A controlled user study assessed skill development in logic puzzles using pre/post-AI assessments, manipulating AI informativeness (1 or 3 objects revealed) and tracking usage behavior.

In practice

• Design AI to complement human cognition.
• Regulate AI access for skill promotion.
• Tailor AI assistance to user ability.

Topics

• Human-AI Interaction
• Skill Development
• AI Informativeness
• Logical Reasoning
• Cognitive Offloading
• User Heterogeneity

Best for: AI Scientist, Research Scientist

Related on AIssential

• Por que estou parando de usar IA(e por que talvez você também deva) — Over-reliance on AI for cognitive tasks may degrade critical thinking and independent learning abilities.
• Uneven Evolution of Cognition Across Generations of Generative AI Models — Generative AI models exhibit uneven cognitive development, excelling in language but struggling with visual-perceptual reasoning.
• The professional judgment gap: Tracing AI’s impact from lecture hall to professional services — AI adoption risks creating a "professional judgment gap" by automating entry-level experiences vital for skill development.
• We destroyed AI. For Good? — Current AI research is overly fragmented, lacking a unified theoretical framework for intelligence.
• The AI paradox problem — As AI automates thought, human cognitive skills risk atrophy, creating an "AI paradox."
• Did AI make me stupid? — Over-reliance on AI for cognitive tasks may lead to a decline in natural memory and problem-solving abilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.