GeoDial: A Multimodal Conversational Tutoring Dataset for Geometry Problem-Solving with Visual Tutor Turns

2026-06-12 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

GeoDial is a new multimodal tutoring dataset comprising over 1.3K teacher-student dialogs focused on geometry problem-solving, collected from experienced math teachers. Unlike most existing datasets, GeoDial explicitly grounds instructional turns in diagram highlights, addressing the limitation of text-only AI tutors. The dataset was created using a scalable annotation protocol integrating dialog acts, visual highlighting, and feedback. Experiments fine-tuning Vision-Language Models (VLMs) like Qwen3-VL-32B on GeoDial show substantial improvements in generating tutoring utterances, but models struggle to produce accurate diagram highlights. This challenge stems from highlight sparsity and the tight coupling between visual actions and teacher utterances, indicating a key area for future research in visually grounded pedagogical modeling.

Key takeaway

For AI Scientists and Machine Learning Engineers developing educational AI, you should prioritize multimodal datasets like GeoDial to advance visually grounded tutoring. Current VLMs show promise in generating pedagogical language but struggle with precise diagram highlighting; therefore, focus your research on improving visual reasoning integration with instructional strategies, potentially by exploring separate highlight prediction models or weighted training approaches to overcome data sparsity.

Key insights

Multimodal tutoring datasets with visual grounding are crucial for developing effective AI tutors.

Principles

• Visual information plays a critical role in learning across subjects.
• AI tutors require multimodal datasets to mimic human teaching effectively.

Method

The GeoDial annotation protocol integrates dialog acts, visual highlighting, and feedback, using VLM-simulated students and VLM-suggested teacher utterances to scale data collection.

In practice

• Fine-tune VLMs on multimodal datasets to improve tutor utterance generation.
• Employ weighted training for highlight generation to address data imbalance.

Topics

• Multimodal Tutoring
• Geometry Problem-Solving
• Vision-Language Models
• Educational Datasets
• Diagram Highlighting
• AI Tutors

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• From Visual Question Answering to multimodal learning: an interview with Aishwarya Agrawal — VQA research has evolved from basic perception to complex multimodal challenges like cultural alignment and embodied AI.
• MM-Conv: A Multimodal Dataset and Benchmark for Context-Aware Grounding in 3D Dialogue — Decoupling linguistic disambiguation from visual localization significantly improves multimodal grounding in dynamic dialogue.
• Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models — Imaginative Perception Tokens (IPT) improve VLM spatial reasoning by externalizing unobserved perceptual information.
• Dial HEALTHDIAL for Advice: A Multilingual and Multi-Parallel Spoken Dialogue Dataset for Knowledge-Grounded Information Seeking — Creating large-scale, multilingual, multi-parallel spoken dialogue datasets is methodologically challenging, revealing performance disparities.
• OphIn-500K: Curating Web-Scale Visual Instructions for Scaling Ophthalmic Multimodal Large Language Models — Curating web-scale ophthalmic video data via a specialized pipeline enables superior domain-specific Multimodal Large Language Models.
• Visual Reasoning Benchmark: Evaluating Multimodal LLMs on Classroom-Authentic Visual Problems from Primary Education — MLLMs struggle with dynamic visual reasoning despite proficiency in static visual tasks, posing risks for educational applications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.