Examining the Usage of Generative AI Models in Student Learning Activities for Software Programming

2026-06-12 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, extended

Summary

A controlled study involving 24 undergraduate students, split into beginner and intermediate programming proficiency groups, investigated the impact of Generative AI (GenAI) tools like ChatGPT on programming task performance and knowledge gains. Participants used either GPT-3.5-turbo or conventional online resources to solve a 30-minute Dynamic Sliding Window programming problem. The research found that generating complete solutions with GenAI significantly boosted task performance, particularly for beginners, but did not consistently translate into improved conceptual understanding. Beginners often exhibited heavy reliance on GenAI for task completion, frequently without knowledge acquisition, whereas intermediate students adopted more selective usage patterns. Critically, both excessive and insufficient GenAI use led to diminished knowledge gains, underscoring the necessity for structured guidance in integrating GenAI into programming curricula.

Key takeaway

For AI Students or Research Scientists integrating Generative AI into programming learning, recognize that uncritical reliance on complete solution generation impedes genuine knowledge acquisition. While GenAI accelerates task completion, prioritize developing a clear conceptual understanding and a personal coding plan. Use GenAI as a learning partner for explanations, bug detection, and test generation. Avoid using it solely as a problem-solver to foster deeper comprehension.

Key insights

GenAI significantly improves programming task performance but often impedes knowledge gains, particularly for beginners who over-rely on it.

Principles

• GenAI exposure timing shapes usage intent.
• Over-reliance on complete GenAI solutions hinders deep learning.
• Balanced GenAI integration is vital for knowledge acquisition.

Method

A $2\times 2$ between-subjects experiment with 24 undergraduate students (beginner/intermediate, with/without ChatGPT) solving a programming task after a concept video, analyzing performance, surveys, and chat histories.

In practice

• Form a clear plan before generating code.
• Use GenAI for conceptual explanations and validation.
• Explore GenAI for bug detection and test generation.

Topics

• Generative AI
• Programming Education
• Student Learning
• ChatGPT
• Knowledge Acquisition
• Learning Strategies

Best for: AI Scientist, AI Student, Research Scientist

Related on AIssential

• 95% of UK students now use AI and their experiences couldn't be more divided — Widespread AI adoption among UK students creates a critical gap in university support and raises concerns about skill development.
• OpenAI upgrades ChatGPT with interactive learning tools as lawsuits and Pentagon backlash mount — Interactive visual tools in ChatGPT enhance math and science learning amidst OpenAI's escalating legal and financial challenges.
• Is Gen Alpha Learning Anything in School, or Just Using AI for Everything? — AI use for homework is widespread among Gen Alpha, disproportionately affecting low-income and minority students.
• The Innovation Advantage GenAI Can’t Give You — Generative AI makes ideas a commodity, shifting innovation advantage to human insight in identifying and reframing core problems.
• The Role of Instructional Guidance in Generative AI-Assisted Learning: Empirical Evidence from Construction Engineering Education — Instructional guidance, like a five-step prompting framework, significantly enhances generative AI's effectiveness for deeper learning tasks.
• Bridging the Gap: Mastering Generative AI in Production with NetCom Learning — Deploying Generative AI in production requires specialized GenAIOps practices beyond basic prompt engineering.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.