TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization

2026-06-11 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

TWLA is a novel post-training quantization (PTQ) framework designed to significantly reduce the memory and compute costs of large language models (LLMs). Published on 2026-06-11, TWLA achieves 1.58-bit weight compression and 4-bit activation quantization, addressing the limitations of existing ternarization methods that struggle with heavy-tailed activation distributions. The framework integrates three key components: Euclidean-to-Manifold Asymmetric Ternary Quantizer (E2M-ATQ) for minimizing layer-output error, Kronecker Orthogonal Tri-Modal Shaping (KOTMS) to reshape weights and suppress activation outliers, and Inter-Layer Aware Activation Mixed Precision (ILA-AMP) for optimizing bit allocation by considering adjacent-layer interactions. Extensive experiments confirm TWLA maintains high accuracy under W1.58A4 while delivering substantial inference acceleration.

Key takeaway

For Machine Learning Engineers deploying LLMs with stringent memory and compute constraints, TWLA offers a critical advancement. This framework allows you to achieve W1.58A4 quantization, significantly reducing model size and accelerating inference without sacrificing accuracy. Consider integrating TWLA's techniques to overcome activation quantization hurdles and optimize your LLM deployments for efficiency and performance.

Key insights

TWLA enables aggressive LLM quantization (W1.58A4) by addressing activation distribution challenges and inter-layer effects.

Principles

• Ternarization significantly compresses LLMs.
• Heavy-tailed activations limit end-to-end acceleration.
• Inter-layer interaction costs prevent quantization cascades.

Method

TWLA employs E2M-ATQ for weight ternarization, KOTMS for reshaping weights and suppressing outliers, and ILA-AMP for mixed-precision bit allocation considering adjacent-layer interactions.

In practice

• Achieve 1.58-bit weight and 4-bit activation quantization.
• Reduce LLM memory and compute costs.
• Enable significant inference acceleration.

Topics

• Large Language Models
• Post-Training Quantization
• Ternary Quantization
• Model Compression
• Low-Bit Activation
• Inference Acceleration

Best for: MLOps Engineer, AI Engineer, NLP Engineer, AI Scientist, Machine Learning Engineer, AI Architect

Related on AIssential

• TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization — TWLA enables 1.58-bit weights and 4-bit activations for LLMs, overcoming heavy-tailed activation challenges for end-to-end acceleration.
• TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization — TWLA enables efficient LLM inference by jointly quantizing weights to 1.58-bit and activations to 4-bit via a three-module PTQ framework.
• STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models — STaR-Quant improves Diffusion LLM efficiency by addressing state-dependent activation disparity and temporal error accumulation in low-bit quantization.
• DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics — DynamicPTQ uses residual-stream dynamics to identify and selectively apply 8-bit activation quantization, improving low-bit LLM inference.
• Channel-Wise Mixed-Precision Quantization for Large Language Models — CMPQ adaptively quantizes LLMs channel-wise using mixed-precision and outlier protection for efficient edge deployment.
• Lost in Quantization: Activation Outliers Explain Language-Specific FP8 Sensitivity in Llama-3 — Activation outliers explain language-specific FP8 quantization sensitivity in LLMs like Llama-3.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.