Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception

2026-07-09 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A study published on 2026-07-09 investigates how deep vision models represent textures, comparing Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and human perception. Diverging from typical object recognition, researchers focused on texture perception using textures generated by three different algorithms from source images. They quantified information in the internal representations of one CNN and three ViTs using a rank-based statistic, then compared these to human psychophysics data. Findings indicate that texture representations align across different ViTs but not between ViTs and the CNN. Furthermore, ViTs form similar representations for textures of varying complexity, and human texture recognition performance is better predicted by ViT representations than CNNs. These results suggest ViTs may more accurately model human texture processing, implying network architecture drives texture representations in computational models.

Key takeaway

For AI Scientists developing vision models focused on texture analysis, you should prioritize Vision Transformers over Convolutional Neural Networks. This research indicates ViTs more faithfully capture human texture processing, offering better predictive power for human recognition tasks. Your model architecture directly impacts texture representation, suggesting a need to favor ViT-based designs for applications requiring human-like texture understanding.

Key insights

Vision Transformers better align with human texture perception than CNNs, driven by architecture.

Principles

• ViT texture representations align across models.
• Network architecture influences texture representation.
• CNNs diverge from human texture processing.

Method

Quantified information in internal representations of one CNN and three ViTs using a rank-based statistic, comparing to human psychophysics data for textures of varying complexity.

In practice

• Prioritize ViTs for texture-centric vision tasks.
• Investigate architectural drivers of perception.

Topics

• Texture Perception
• Vision Transformers
• Convolutional Neural Networks
• Human Psychophysics
• Deep Vision Models
• Computational Vision Science

Best for: Computer Vision Engineer, AI Scientist, Research Scientist

Related on AIssential

• Vision Transformers Learn Gestalt-Like Figure-Ground Cues from Natural Images — Vision Transformers acquire Gestalt-like figure-ground organization cues directly from natural image statistics.
• AI doesn’t ‘see’ the way that you do, and that could be a problem when it categorizes objects and scenes — AI vision systems misclassify objects due to reliance on surface textures, not human-like shape and context.
• Computer Vision : Chapter 1 — Convolutional Neural Networks evolved from biological vision models and mathematical convolution to automatically learn hierarchical image features.
• Vision Models in AI: How Machines Learn to See the World — Vision models have evolved from CNNs to multimodal VLMs, enabling machines to interpret and generate visual data with semantic understanding.
• Sidewalk Moments: Are Richer Representations Always More Human-Aligned? Evidence from City-Walk Videos — Richer visual representations do not inherently guarantee greater human alignment for urban engagement tasks.
• Masked Autoencoder for Image Reconstruction using Vision Transformers (PyTorch) — Masked Autoencoders with Vision Transformers learn robust image representations by reconstructing heavily masked image patches.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.