We designed an AI tutor that helps college students reason rather than give them answers

2026-03-05 · Source: Artificial intelligence (AI) – The Conversation · Field: Education & Learning — Educational Technology (EdTech), Academic Research & Higher Education, Educational Psychology & Learning Sciences · Depth: Intermediate, short

Summary

A study conducted in spring 2025 with 140 undergraduate macroeconomics students at the University of Wisconsin, La Crosse, investigated the impact of AI-powered tutoring on learning outcomes. Researchers developed "Macro Buddy," a custom ChatGPT tool with web access disabled, trained on course-specific materials to guide students through reasoning rather than providing direct answers. The study found that students who used Macro Buddy in conjunction with peer discussion achieved higher exam scores than those who studied alone. Students using Macro Buddy individually also scored higher than a control group working alone without AI. This suggests that AI, when designed as a Socratic tutor and combined with collaborative learning, can enhance student understanding and performance, challenging concerns about AI eroding learning.

Key takeaway

For AI Scientists developing educational tools, focus on designing AI that acts as a Socratic tutor, prompting students with questions rather than delivering immediate answers. Your design choices, such as disabling web access and integrating with peer learning, can significantly influence whether AI supports or replaces critical thinking. Prioritize interactive guidance and collaborative features to maximize learning efficacy and mitigate concerns about AI eroding core learning skills.

Key insights

AI designed as a Socratic tutor, combined with peer discussion, enhances student learning outcomes.

Principles

• AI as a guide fosters deeper learning.
• Peer discussion enhances reasoning articulation.
• Learning sticks when the work is hard.

Method

A custom GPT, Macro Buddy, was trained on course materials and designed to ask guiding questions instead of providing direct answers, then integrated into various study formats for undergraduate macroeconomics students.

In practice

• Design AI tutors to ask guiding questions.
• Combine AI tutoring with peer collaboration.
• Disable web access for focused AI learning.

Topics

• AI in Education
• Generative AI
• AI Tutors
• Learning Outcomes
• Educational Technology

Best for: AI Scientist, Research Scientist, AI Chatbot Developer, Policy Maker

Related on AIssential

• This AI Taught Me Calculus in 5 Minutes — A multi-agent AI tutor provides Socratic guidance and interactive learning, surpassing traditional answer-giving AI.
• Pupils in England are losing their thinking skills because of AI, survey suggests — AI use by students may diminish critical thinking and core skills, while teachers increasingly adopt AI for administrative tasks.
• Measuring the impact of learning with AI in Sierra Leone and beyond — AI, when designed for "Socratic" scaffolding, significantly improves student math learning and engagement without replacing teachers.
• How Teachers Help Students Show Their Thinking with AI — AI can scale individualized student assessment by capturing reasoning through conversational dialogue.
• How Does AI Support Education? Tools That Improve Learning — AI in education personalizes learning, automates routine tasks, and provides immediate support, enhancing efficiency for both students and teachers.
• Beyond Passive Viewing: A Pilot Study of a Hybrid Learning Platform Augmenting Video Lectures with Conversational AI — Integrating conversational AI tutors with video lectures significantly boosts learning outcomes and engagement in online education.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial intelligence (AI) – The Conversation.