Using street view images and visual LLMs to predict heritage values for governance support: Risks, ethics, and policy implications

2025-12-22 · Source: cs.AI updates on arXiv.org · Field: Government & Public Sector — Public Policy & Governance, Civic Technology & Smart Cities · Depth: Expert, quick

Summary

A research initiative aimed to assist Swedish authorities in developing information on heritage values within the national building stock, a critical need for the 2025-2026 implementation of the EU's Energy Performance of Buildings Directive. Lacking a comprehensive national register, Sweden faces a barrier in creating National Building Renovation Plans. The study analyzed 154,710 street view images from across Sweden using multimodal Large Language Models (LLMs) to perform zero-shot predictions, identifying buildings with potential heritage values across 5.0 million square meters of heated floor area. The findings, lessons learned, and methodological challenges are presented, alongside a discussion of risks associated with authorities using LLM-based data, particularly concerning transparency, error detection, and sycophancy in governance contexts.

Key takeaway

For urban planners or policy makers developing national building renovation plans, you must critically evaluate LLM-generated heritage value data. Prioritize robust error detection and transparency mechanisms to mitigate risks like sycophancy and ensure public trust. Consider integrating human expert review to validate LLM predictions before making policy decisions impacting historical preservation.

Key insights

Visual LLMs analyzing street view images can predict building heritage values for governance, but raise significant risks regarding transparency and error.

Principles

• LLM-based data use by authorities risks transparency.
• Error detection is critical for LLM outputs in governance.
• LLM sycophancy can bias predictive models.

Method

Multimodal LLMs performed zero-shot predictions on 154,710 street view images across Sweden to assess visible aspects indicative of building heritage value for governance support.

In practice

• Identify buildings for renovation plans.
• Support national heritage registers.
• Inform urban planning decisions.

Topics

• Visual LLMs
• Heritage Preservation
• Urban Planning
• Policy Implications
• AI Ethics
• Street View Data
• Building Renovation

Best for: AI Scientist, Policy Maker, AI Ethicist, Research Scientist

Related on AIssential

• Leveraging Multimodal LLMs for Built Environment and Housing Attribute Assessment from Street-View Imagery — Multimodal LLMs, fine-tuned and distilled, can automate large-scale building condition assessment from street-view imagery with high accuracy and efficiency.
• From Benchmarking to Reasoning: A Dual-Aspect, Large-Scale Evaluation of LLMs on Vietnamese Legal Text — LLMs struggle with legal reasoning and application, often masking critical errors despite high surface-level accuracy.
• Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research — LLMs offer transformative potential for low-resource language research in humanities, despite significant data and ethical challenges.
• What is the scientific value of administering the standard Rorschach test to LLMs when the training data is almost certainly contaminated? (R) + [D] — Testing LLMs with widely available psychometric stimuli primarily assesses data retrieval, not genuine perception.
• The Anatomy of an LLM Benchmark — Effective LLM benchmarking requires continuous evolution, rigorous data quality, and dynamic evaluation methods to counter rapid model saturation.
• On the Cultural Anachronism and Temporal Reasoning in Vision Language Models — VLMs misinterpret historical artifacts due to "cultural anachronism," applying temporally inappropriate modern concepts.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.