What Matters in Practical Learned Image Compression

2026-05-07 · Source: Apple Machine Learning Research · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Internet of Things (IoT) & Connected Devices · Depth: Expert, quick

Summary

Apple researchers have developed a new learned image codec, optimized for both perceptual quality and on-device runtime, aiming to bridge the gap between potential and practical application of learned codecs. The study involved a comprehensive analysis of key modeling choices and novel techniques, followed by a performance-aware neural architecture search across millions of configurations. This process identified models that meet specific on-device runtime targets while maximizing compression performance based on perceptual metrics. The resulting codec demonstrates significant improvements, achieving 2.3–3x bitrate savings compared to traditional codecs like AV1, AV2, VVC, ECM, and JPEG-AI, and 20–40% savings against leading learned codecs. Notably, it encodes 12MP images on an iPhone 17 Pro Max in 230ms and decodes them in 150ms, outperforming many ML-based codecs running on a V100 GPU.

Key takeaway

For AI Engineers developing mobile-first image processing applications, this research indicates that jointly optimizing for perceptual quality and on-device runtime can yield substantial performance gains. You should explore neural architecture search techniques to identify codec configurations that meet specific hardware constraints, as demonstrated by the 230ms encode and 150ms decode times for 12MP images on an iPhone 17 Pro Max, while also achieving significant bitrate savings.

Key insights

Optimizing learned image codecs for both perceptual quality and on-device speed yields superior compression and performance.

Principles

• Joint optimization improves codec tradeoffs.
• Neural architecture search refines model efficiency.

Method

A comprehensive study of modeling choices, including novel techniques, followed by performance-aware neural architecture search to identify optimal backbone configurations for target runtime and perceptual quality.

In practice

• Achieve 2.3–3x bitrate savings over AV1.
• Encode 12MP images in 230ms on mobile.
• Decode 12MP images in 150ms on mobile.

Topics

• Learned Image Compression
• Perceptual Quality Optimization
• Neural Architecture Search
• On-Device Runtime
• Bitrate Savings

Code references

• apple/ml-pico

Best for: AI Engineer, Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Hardware Engineer

Related on AIssential

• Learned JPEG Compression for DNN Vision — J4D optimizes JPEG for DNNs using a differentiable codec and rate estimator, significantly boosting accuracy or reducing file size.
• An Extensible and Lightweight Unified Architecture for Demosaicing Pixel-bin Image Sensors — A unified, lightweight architecture with CFA identification improves pixel-bin sensor demosaicing quality and efficiency.
• Custom image encoder [P] — Tailoring image encoders for specific video classification tasks can significantly improve performance on resource-constrained devices.
• AdaCodec: A Predictive Visual Code for Video MLLMs — AdaCodec uses a predictive visual code to reduce redundancy in video MLLMs, improving efficiency and performance.
• Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression — Ultra-low bitrate image compression can achieve pixel-level fidelity by combining semantic and pixel representations in a diffusion model.
• From Pixels to DNA: Why the Future of Compression Is About Every Kind of Data — Data compression is evolving into a foundational technology for all digital data, shifting from human-centric media to machine-centric semantics.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Apple Machine Learning Research.