Automatically Inferring Teachers' Geometric Content Knowledge: A Skills Based Approach

2026-04-15 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

A new study introduces an automated method for assessing teachers' geometric content knowledge, specifically their Van Hiele reasoning levels, using large language models. Traditional assessment of these five hierarchical levels relies on time-consuming manual expert analysis of open-ended responses. Researchers developed a structured skills dictionary, breaking down Van Hiele levels into 33 fine-grained reasoning skills. They collected 226 geometry problem responses from 31 pre-service teachers via a custom web platform, which were then expert-annotated with Van Hiele levels and demonstrated skills. Two classification approaches, retrieval-augmented generation (RAG) and multi-task learning (MTL), were implemented. Both approaches showed that skills-aware variants, which incorporated the skills dictionary, significantly outperformed baselines lacking skills information across various evaluation metrics. This work represents the first automated Van Hiele level classification from open-ended responses.

Key takeaway

For educational technologists or researchers developing scalable teacher assessment tools, this study demonstrates that incorporating a structured skills dictionary significantly boosts the accuracy of automated Van Hiele level classification. You should consider integrating fine-grained skill decomposition into your LLM-based assessment models to improve diagnostic precision and enable more effective personalized learning systems for educators.

Key insights

Integrating explicit skills information significantly improves automated Van Hiele level classification for geometric content knowledge.

Principles

• Van Hiele model defines five hierarchical geometric reasoning levels.
• Automated assessment can scale teacher content knowledge evaluation.

Method

The method involves creating a skills dictionary, collecting expert-annotated open-ended responses, and applying skills-aware RAG or MTL models for classification.

In practice

• Use skills dictionaries to enhance LLM classification accuracy.
• Apply RAG or MTL for educational assessment automation.

Topics

• Van Hiele Model
• Geometric Content Knowledge
• Large Language Models
• Skills-Based Assessment
• Retrieval-Augmented Generation

Best for: AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• Automatically Inferring Teachers' Geometric Content Knowledge: A Skills Based Approach — Integrating explicit skills information significantly improves LLM-based classification of teachers' Van Hiele geometric reasoning levels.
• Constructing Evaluation Datasets for Procedural Reasoning: Balancing Naturalness, Grounding, and Multi-Hop Coverage — Effective AI procedural reasoning evaluation requires datasets explicitly grounded in structured instructional knowledge.
• Concept Extraction and Webb’s Depth of Knowledge: Comparing LLM Question Generation Pipelines for Educational Assessment — Comparing LLM pipelines for educational question generation using concept extraction and Webb's Depth of Knowledge.
• Human-in-the-Loop Benchmarking of Heterogeneous LLMs for Automated Competency Assessment in Secondary Level Mathematics — LLM architecture compatibility with instruction constraints is more critical than model scale for rubric-constrained assessment tasks.
• ScholarSum: Student-Teacher Abstractive Summarization via Knowledge Graph Reasoning and Reflective Refinement — ScholarSum integrates hierarchical knowledge graphs and iterative student-teacher refinement for fluent, factually faithful scientific summarization.
• Evaluation of Image Matching for Art Skills Assessment — Automated drawing skill assessment is feasible using computer vision, with SIFT-based key point matching proving effective for hand-drawn image comparison.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.