Two-Valued Symmetric Circulant Matrices: Applications in Deep Learning

2026-05-19 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Internet of Things (IoT) & Connected Devices, Health & Medical Research · Depth: Expert, extended

Summary

A new deep learning architecture, the Two-Valued Symmetric Circulant Matrix (TVSCM), is proposed to address the high storage and computational demands of deep neural networks, particularly for resource-limited edge and Internet of Medical Things (IoMT) platforms. This architecture replaces traditional dense layers with a highly sparse structure that uses only two weights per layer, arranged in a circulant and symmetric configuration. Experimental results on the MNIST dataset show an approximately 79x reduction in parameters (from 623,290 to 7,852) and an 80.4x reduction in memory access, while maintaining comparable accuracy (97.67% to 93.54%). On the MIT-BIH arrhythmia dataset, TVSCM achieved a 26x parameter reduction (from 24,709 to 942) and a 26.2x memory access reduction, with accuracy shifting from 97.60% to 93.10%. The TVSCM model also demonstrated significant improvements in energy efficiency (50x on MNIST, 112x on MIT-BIH) and inference latency (117x on MNIST, 91.5x on MIT-BIH), making it suitable for battery-powered and real-time healthcare applications.

Key takeaway

For AI Engineers and Research Scientists developing models for edge or IoMT platforms, consider integrating Two-Valued Symmetric Circulant Matrices (TVSCM) into your network architecture. This approach can yield significant reductions in model parameters (up to 80x), memory footprint, energy consumption, and inference latency, making your models viable for resource-constrained environments without substantial accuracy loss. Prioritize TVSCM for applications requiring real-time processing and extended battery life, such as wearable health monitors or embedded diagnostic tools.

Key insights

Two-Valued Symmetric Circulant Matrices drastically reduce deep learning model parameters and computational overhead for edge devices.

Principles

• Structured sparsity enables extreme parameter reduction.
• Circulant and symmetric matrices simplify weight representation.
• FFT-based computations accelerate matrix operations.

Method

Replace dense neural network layers with Two-Valued Symmetric Circulant Matrices (TVSCM) that learn an entire weight matrix from just two scalar values, leveraging cyclic shifts and symmetry for compression.

In practice

• Deploy TVSCM for lightweight inference on IoMT devices.
• Utilize TVSCM to reduce energy consumption in battery-powered systems.
• Apply TVSCM for real-time anomaly detection on edge platforms.

Topics

• Two-Valued Symmetric Circulant Matrix
• Deep Learning Model Compression
• Edge Computing
• Internet of Medical Things
• Parameter Reduction

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Hardware Engineer

Related on AIssential

• Uncovering Symmetry Transfer in Large Language Models via Layer-Peeled Optimization — LLM training implicitly transfers data symmetries into structured geometric configurations within model parameters.
• Sample efficient inductive matrix completion with noise and inexact side information — Noisy inductive matrix completion can achieve reduced sample complexity and sharp error guarantees simultaneously.
• Quantization Effects of Artificial Neural Networks for Embedded Edge-Computing Applications — LUT-based 2-bit BNNs trained with GAs enable sub-50ns real-time waveform classification on FPGAs.
• Expressivity of congruence-based architectures for DNNs on positive-definite matrices — Semi-orthogonality in congruence layers limits DNN expressivity for SPD matrix classification due to spectral diversity loss.
• Neural-Guided Domain Restriction to Accelerate Pseudospectra Computation for Structured Non-normal Banded Matrices — A neural network predicts sensitive regions to accelerate pseudospectra computation for non-normal matrices.
• Preconditioned inexact stochastic ADMM for deep models — PISA is a new ADMM-based optimizer for deep learning, offering robust convergence and superior performance on heterogeneous data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.