AI vs Human Expert Reasoning: Assessing Agreements in Building Typology Predictions based on Street View Imagery

2026-07-16 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, quick

Summary

This research evaluates Vision-Language Models (VLMs) for inferring building typologies—Construction, Current Use, and Storeys—from Google Street View (GSV) images. The study compares VLM predictions from models like GPT-4o, Claude 3.5 Sonnet, and Gemini 2.0 Flash against human expert inferences by civil engineers and architects. It found that Chain-of-Thought prompts generally provide more stable model performance. Analyzing keyword probabilities in AI explanations revealed that VLMs primarily focus on visual indicators, while human experts integrate broader contextual cues and domain knowledge. Overall, VLMs can approximate human expert classification capabilities at scale, achieving an average accuracy of approximately 70%, demonstrating their potential for AI automation in urban analysis tasks requiring visual pattern recognition.

Key takeaway

For urban analysts or civil engineers considering AI for large-scale building typology classification, Vision-Language Models like GPT-4o can achieve approximately 70% accuracy compared to human experts. You should explore integrating these VLMs, particularly with Chain-of-Thought prompting, to automate initial pattern recognition from Street View imagery. This frees up expert time for nuanced contextual analysis, allowing for scalable, complementary urban analysis workflows.

Key insights

VLMs can approximate human expert building typology classification from Street View images with ~70% accuracy, focusing on visual indicators.

Principles

• Chain-of-Thought prompts stabilize VLM performance.
• AI prioritizes visual indicators; humans use broader context.
• VLMs offer scalable pattern recognition for urban analysis.

Method

VLMs (GPT-4o, Claude 3.5 Sonnet, Gemini 2.0 Flash) infer building typologies from GSV images, comparing predictions with human expert labels, and analyzing reasoning via keyword probabilities.

In practice

• Automate urban analysis tasks requiring visual identification.
• Complement human experts in urban planning.
• Utilize Chain-of-Thought for VLM stability.

Topics

• Vision-Language Models
• Building Typology
• Google Street View
• Urban Analysis
• Chain-of-Thought Prompting
• AI Automation

Best for: Computer Vision Engineer, AI Scientist, Research Scientist, Machine Learning Engineer

Related on AIssential

• AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring — AVA-VLM uses adaptive visual attention and coarse-to-fine reasoning to enhance VLM reliability and efficiency for construction site monitoring.
• Can AI Reason Like an Urban Planner? Benchmarking Large Language Models Against Professional Judgment — The UPBench framework reveals LLMs struggle with context-sensitive, lower-order urban planning tasks despite performing well on higher-order analysis.
• Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models — VLMs struggle with human-like spatial deictic expression usage, particularly distance-based demonstratives, across languages.
• From Codebooks to VLMs: Evaluating Automated Visual Discourse Analysis for Climate Change on Social Media — VLMs can effectively analyze social media images for climate discourse, even with moderate per-image accuracy.
• Leveraging Multimodal LLMs for Built Environment and Housing Attribute Assessment from Street-View Imagery — Multimodal LLMs, fine-tuned and distilled, can automate large-scale building condition assessment from street-view imagery with high accuracy and efficiency.
• Benchmarks for Vision-Language Models in Urban Perception Should Be Reliability-Aware and Negotiated — Benchmarking VLMs for urban perception must account for human disagreement and abstention, treating them as valid measurement outcomes.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.