WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution

2026-07-02 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

Windowed Batch Matrix Multiplication (WBMM) is a novel technique designed to overcome the performance degradation of large kernel depthwise convolutions, which suffer from irregular memory access. While Large Kernel Acceleration (LKA) helps small feature maps, it becomes counterproductive on larger ones. WBMM addresses this by partitioning input into contiguous windows and using a compact relative position bias table to construct weight matrices, enabling regular memory access via batched matrix multiplication. This approach uniquely allows WBMM's throughput to improve with larger windows, contrasting with depthwise convolutions. Operator-level benchmarks show WBMM with 14x14 windows outperforms 5x5 depthwise convolution baselines in speed, providing a 7.8x larger per-layer receptive field. Combined with inter-block cross-window communication and hierarchical window reparameterization, WBMM achieves comparable or higher accuracy on ImageNet-1K, COCO, and ADE20K, with a 1.31-1.88x training speedup across GPU, CPU, and edge devices.

Key takeaway

For Computer Vision Engineers developing models requiring large receptive fields, WBMM offers a significant performance advantage. You should consider integrating WBMM to overcome the efficiency issues of traditional large kernel depthwise convolutions. This approach provides a 1.31-1.88x training speedup and maintains high accuracy across diverse hardware, making it a robust alternative for improving model efficiency and performance in tasks like image classification and segmentation.

Key insights

WBMM uses windowed batch matrix multiplication to enable efficient large receptive field convolutions with regular memory access.

Principles

• Large kernel depthwise convolutions degrade with size.
• Regular memory access improves convolution throughput.
• Windowing can reverse performance scaling with kernel size.

Method

WBMM partitions input into contiguous windows, indexes a compact relative position bias table to construct weight matrices, then performs batched matrix multiplication for regular memory access.

In practice

• Use WBMM for large receptive field models.
• Apply WBMM for training speedup on various devices.
• Integrate WBMM for improved accuracy on vision tasks.

Topics

• WBMM
• Depthwise Convolutions
• Large Kernel Convolutions
• Receptive Field
• Batched Matrix Multiplication
• Computer Vision
• GPU Acceleration

Code references

• wansong-s/WBMM

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Accelerating Multimodal Inference in vLLM: The One-Line Optimization for Large Multimodal Models — Batch-level Data Parallelism in vLLM significantly boosts multimodal inference by reducing vision encoder communication overhead.
• Scalable Operator Learning via Nystr\"om Approximation With Denoising Applications — Nyström subsampling scales operator learning in vRKHS, achieving optimal convergence and efficient functional output prediction.
• ZipMap: Linear-Time Stateful 3D Reconstruction with Test-Time Training — ZipMap enables linear-time 3D reconstruction by zipping image collections into a compact scene state via test-time training.
• HiRo: A Compact Four-Directional Hierarchical Reservoir Token-Mixer for Efficient Image Classification — HiRo combines hierarchical reservoir computing and shifted windows for efficient, high-accuracy image classification with few parameters.
• Vision Hopfield Memory Networks — V-HMN uses hierarchical Hopfield memory and predictive-coding-inspired refinement for data-efficient, interpretable vision backbones.
• From Spatial to Spectral: An Efficient, Frequency-Guided Feature Representation Learner for Small Object Detection — Shifting small object detection from spatial to spectral processing improves efficiency and accuracy by preserving high-frequency features.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.