LUMI: Tokenizer-Agnostic LLM-Based Lossless Image Compression

2026-07-09 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Image Processing & Compression · Depth: Expert, medium

Summary

LUMI (LLM-based Unified Model-agnostic lossless Image compression) is a novel framework for lossless RGB image compression that operates independently of traditional text tokenizers. Unlike prior LLM-based methods that convert pixel data into text tokens, LUMI employs a pixel embedding module to map raw intensity and channel information directly into the LLM's continuous embedding space. It integrates intra-patch position encoding to preserve 2D spatial structure and utilizes a 256-way prediction head for native pixel alphabet probabilities. The framework trains only these specific modules, keeping the LLM backbone (e.g., LLaMA, Qwen, Gemma) frozen. Experiments across natural, medical, and remote-sensing image benchmarks demonstrate LUMI's competitive compression rates and enhanced cross-domain robustness compared to tokenizer-based LLM compression baselines.

Key takeaway

For Machine Learning Engineers developing image compression solutions, LUMI offers a robust alternative to tokenizer-dependent methods. If you are seeking to leverage large language models for lossless image compression, consider adopting LUMI's pixel-space adaptation approach. This framework allows you to utilize frozen LLM backbones like LLaMA or Gemma, achieving competitive compression rates and significantly improving cross-domain robustness for diverse image datasets, including medical and remote-sensing.

Key insights

LUMI enables tokenizer-agnostic, LLM-based lossless image compression by directly embedding pixels into the model's continuous space.

Principles

• LLMs can estimate image coding probabilities effectively.
• Decoupling image compression from text tokenizers enhances robustness.
• Pixel-space adaptation of frozen foundation models is viable.

Method

LUMI replaces pixel-as-text tokenization with a pixel embedding module, adds intra-patch position encoding, and uses a 256-way prediction head, training only these components while freezing the LLM backbone.

In practice

• Implement LUMI for lossless RGB image compression.
• Utilize LLaMA, Qwen, or Gemma as frozen LLM backbones.
• Improve compression robustness across diverse image domains.

Topics

• LLM-based Compression
• Lossless Image Compression
• Tokenizer-Agnostic
• Pixel Embedding
• Foundation Models
• LLaMA
• Qwen

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• LUMI: Tokenizer-Agnostic LLM-Based Lossless Image Compression — LUMI enables tokenizer-agnostic lossless image compression by adapting frozen LLMs directly in pixel space, enhancing robustness.
• Lossless Prompt Compression via Dictionary-Encoding and In-Context Learning: Enabling Cost-Effective LLM Analysis of Repetitive Data — LLMs can learn dictionary encodings in-context, enabling lossless prompt compression without fine-tuning and preserving analytical accuracy.
• Quantization-Robust LLM Unlearning via Low-Rank Adaptation — Low-rank adaptation (LoRA) enables effective LLM unlearning that survives aggressive post-training quantization.
• End-to-End Context Compression at Scale — LCLMs are encoder-decoder compressors that significantly improve long-context LLM inference efficiency and performance.
• VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs — VarRate compresses KV cache by assigning variable low-rank budgets based on query salience, retaining all tokens.
• Joint Lossless Compression and Steganography for Medical Images via Large Language Models — Joint lossless compression and steganography for medical images utilizes LLMs and VAEs for superior performance and security.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.