Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators

2026-04-16 · Source: cs.AI updates on arXiv.org · Field: Education & Learning — Educational Technology (EdTech), Educational Psychology & Learning Sciences, Academic Research & Higher Education · Depth: Expert, extended

Summary

A study with 432 undergraduate students from the University of Florida investigated how trust in an AI assistant influences appropriate reliance during programming problem-solving tasks. Participants completed 14 Python output-prediction problems, receiving recommendations and explanations from an AI chatbot (gpt-3.5-turbo-0125), with 6 problems containing intentionally misleading suggestions and 8 containing accurate ones. Researchers measured appropriate reliance as the proportion of trials where students accepted correct recommendations and rejected incorrect ones. Pre- and post-task surveys assessed trust, AI literacy, need for cognition, programming self-efficacy, and programming literacy. Results showed a non-linear, negative relationship: higher trust correlated with lower appropriate reliance, suggesting weaker discrimination between correct and incorrect AI suggestions. This relationship was significantly moderated by students' AI literacy and need for cognition, indicating these characteristics influence how trust impacts reliance.

Key takeaway

For AI Scientists and Research Scientists developing educational AI tools, these findings highlight that simply building trust is insufficient; you must also design for critical engagement. Implement cognitive forcing functions, such as requiring students to commit to an answer before seeing AI recommendations or justifying their choices, to encourage reflective evaluation. This approach can mitigate overreliance, even among students with high AI literacy or a strong need for cognition, fostering more effective learning outcomes.

Key insights

Higher trust in AI can lead to less critical evaluation of its outputs, especially in educational settings.

Principles

• Trust in AI can drive overacceptance of its outputs.
• Automation bias can lead users to favor AI suggestions.
• Cognitive offloading reduces metacognitive monitoring.

Method

A "Wizard of Oz" experiment with 432 undergraduates used a Python programming task with an AI chatbot providing mixed correct and misleading recommendations. Reliance was behaviorally measured by acceptance/rejection of suggestions.

In practice

• Design learning environments to make AI evaluation explicit.
• Embed cognitive forcing functions into AI interactions.
• Prompt students to justify agreement/disagreement with AI.

Topics

• AI in Education
• Human-AI Trust
• Appropriate Reliance
• AI Literacy
• Need for Cognition

Best for: AI Scientist, Research Scientist

Related on AIssential

• Choosing to Stay Human — Intentional AI use, rather than default reliance, is critical to prevent cognitive surrender and foster genuine learning.
• AI Literacy at Scale: K-to-Career Access That Delivers Real Student Outcomes — AI integration in education requires balancing student engagement with responsible use, curriculum adaptation, and equitable access.
• Enabling and Inhibitory Pathways of University Students' Willingness to Disclose AI Use: A Cognition-Affect-Conation Perspective — Psychological safety and clear institutional guidance are key to students' willingness to disclose AI use.
• AI is becoming a second brain at the expense of your first one — Over-reliance on AI can erode human judgment, leading to disempowerment and the adoption of AI's biases.
• The consequences of relying on AI for accurate news — Over-reliance on AI for information verification can degrade human discernment skills, creating an "AI dependency paradox."
• Over half of Americans fear losing both their jobs and their independent thinking to AI, survey finds — Public perception of AI is split between high hopes for health advancements and significant fears of job displacement and cognitive reliance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.