Micro-Diffusion Compression -- Binary Tree Tweedie Denoising for Online Probability Estimation

2026-03-11 · Source: stat.ML updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, long

Summary

Midicoth is a novel lossless compression system that integrates "micro-diffusion," a multi-step score-based reverse diffusion process, into a cascaded statistical modeling pipeline. This system employs Tweedie's empirical Bayes formula to reverse the shrinkage effect of Jeffreys-prior smoothing, which pulls empirical distributions toward uniform. It achieves this through a binary tree decomposition, breaking down 256-way byte predictions into eight binary decisions (MSB to LSB), with additive Tweedie corrections estimated nonparametrically across three denoising steps. The Midicoth pipeline consists of five online, parameter-free layers: an adaptive PPM model (orders 0-4), an extended match model, a trie-based word model, a high-order context model (orders 5-8), and the micro-diffusion layer as a final post-blend correction. Midicoth achieves 1.753 bpb on enwik8, outperforming xz -9 by 11.9%, and 2.119 bpb on alice29.txt, surpassing xz -9 by 16.9%, all without neural networks, training data, or GPUs.

Key takeaway

For AI Scientists developing or evaluating lossless compression algorithms, Midicoth demonstrates that purely statistical, CPU-based methods can achieve competitive performance against dictionary-based compressors and narrow the gap to neural network-based systems. You should consider integrating Tweedie denoising and binary tree decomposition into your adaptive context models to improve prediction sharpness and correct systematic biases, especially when GPU resources or pre-trained models are not feasible.

Key insights

Micro-diffusion with binary tree Tweedie denoising enhances lossless compression by correcting systematic biases in blended probability distributions.

Principles

• Jeffreys smoothing acts as a shrinkage operator.
• Tweedie's formula approximates optimal denoising.
• Multi-step denoising refines score estimates.

Method

Midicoth uses a five-layer cascaded pipeline, applying binary tree Tweedie denoising as a post-blend correction. It decomposes 256-way predictions into binary decisions, estimates additive corrections nonparametrically via calibration tables, and applies them over three steps.

In practice

• Decompose complex predictions into binary decisions for data efficiency.
• Apply post-prediction correction to refine blended model outputs.
• Use James-Stein shrinkage for noisy calibration bins.

Topics

• Statistical Compression
• Micro-Diffusion
• Tweedie Denoising
• Binary Tree Decomposition
• Context Modeling

Best for: AI Scientist, AI Researcher, Machine Learning Engineer, Research Scientist

Related on AIssential

• Colored Noise Diffusion Sampling — Diffusion model SDE inference can be optimized by dynamically allocating noise energy based on frequency resolution.
• TreeGRNG: Binary Tree Gaussian Random Number Generator for Efficient Probabilistic AI Hardware — TreeGRNG provides an energy-efficient and accurate Gaussian random number generation method for edge Bayesian Neural Networks using constant comparators.
• Smoothness-Based Derandomization of PAC-Bayes Bounds — PAC-Bayes derandomization for smooth loss functions quantifies the cost of deterministic predictors using flatness metrics.
• Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission — DDM-SSCC enables lossless pixel-level image transmission by adapting discrete diffusion models for synchronized arithmetic coding.
• Approximate Structured Diffusion for Sequence Labelling — Combining discrete diffusion with a Mean-Field approximated CRF denoiser improves sequence labelling accuracy and scalability.
• [P] Micro Diffusion — Discrete text diffusion in ~150 lines of pure Python — Discrete text diffusion generates all tokens at once by iteratively unmasking from noise.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by stat.ML updates on arXiv.org.