Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models

2026-07-26 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, quick

Summary

Omni-Prune is a novel training-free, query-aware audio-visual token pruning framework designed to enhance the efficiency of Omnimodal Large Language Models (OmniLLMs). These models, which integrate synchronized audio and video, often suffer from high prefill latency and GPU memory consumption due to extensive token sequences. Unlike prior vision-only pruning methods, Omni-Prune explicitly accounts for cross-modal connections between audio and video, alongside user query relevance. The framework operates by segmenting token sequences into adaptive time windows based on audio saliency peaks, then scores audio and video tokens using a unified scale that combines encoder attention with text-query relevance. It also pairs related audio-video tokens to ensure co-retention and employs a K-medoids step within each window to select diverse, representative tokens. Experiments show Omni-Prune achieves up to 3.25x prefill speedup and 1.3x memory reduction, while maintaining over 99% of the full-model's performance.

Key takeaway

For AI Engineers optimizing Omnimodal LLMs, Omni-Prune offers a critical solution to high inference costs. If your models struggle with prefill latency or GPU memory due to long audio-video sequences, you should evaluate this training-free framework. It promises up to 3.25x prefill speedup and 1.3x memory reduction while preserving over 99% performance, enabling more efficient deployment of complex multimodal systems.

Key insights

Omni-Prune efficiently prunes audio-visual tokens in Omnimodal LLMs by integrating query-awareness and cross-modal links.

Principles

• Pruning must consider cross-modal links and user queries.
• Combine encoder attention with text-query relevance for scoring.
• K-medoids can select diverse, representative tokens.

Method

Omni-Prune splits token sequences into adaptive time windows at audio saliency peaks, scores tokens by combining encoder attention with text-query relevance, pairs related audio-video tokens, and uses K-medoids for final selection.

In practice

• Achieve 3.25x prefill speedup for OmniLLMs.
• Reduce GPU memory usage by 1.3x.
• Retain over 99% of full-model performance.

Topics

• Omnimodal LLMs
• Token Pruning
• Audio-Visual Processing
• Efficient Inference
• GPU Memory Optimization
• Query-Aware Pruning

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning — Relevance-anchored contextual expansion effectively prunes visual tokens for efficient multimodal inference in large vision-language models.
• MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models — Modality-aware quantization with bias compensation and morphology-directed optimization significantly improves 4-bit OLLM performance.
• Structured Pruning of Large Language Models via Power Transformation and Sign-Preserving Score Aggregation with Adaptive Feature Retention — Improved structured pruning for LLMs maintains accuracy while enabling practical inference speedups by addressing AFR adaptation challenges.
• How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A — Visual token pruning in MLLMs is best approached as task-conditioned evidence search, not static ranking.
• Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning — EADP improves VLM efficiency by pruning visual tokens, combating textual noise and feature fragmentation with entropy and submodular maximization.
• Doing More With Less: Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling — Unstructured pruning can enhance LLM test-time scaling performance, challenging prior assumptions about pruning's detrimental effects.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.