Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

2026-07-22 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

LeakyLMs introduces a novel set of attacks capable of extracting proprietary model architecture and deployment details from production language models solely by analyzing token generation timing via remote APIs. This work is the first to demonstrate such inference capabilities. The system comprises two primary attacks. One attack identifies inference optimizations and deployment strategies, such as detecting speculative decoding and determining the draft model's context length. For instance, Google Gemini Flash 2.5 was found to employ speculative decoding with a draft context window of approximately 128K tokens. The second attack recovers key architectural properties, including the number of transformer layers, hidden dimension size, and attention heads. This is achieved by building a precise model of token-generation timing on modern NVIDIA GPUs, which characterizes latency scaling with model configuration, and then performing a search over the architecture space. Experiments with Llama models showed the near-correct architectural configuration appearing in the top-10 guesses over 90% of the time.

Key takeaway

For AI Security Engineers and ML Engineers deploying proprietary language models, this research highlights a critical new attack vector. Your production LLMs are vulnerable to architectural and optimization leakage via per-token timing analysis, even through remote APIs. You should implement timing obfuscation techniques or monitor for suspicious timing-based queries to protect sensitive model details like speculative decoding usage or transformer layer counts. This necessitates re-evaluating current API security postures.

Key insights

Proprietary LLM architecture and deployment details can be inferred from per-token generation timing via remote APIs.

Principles

• Token generation timing reveals underlying LLM optimizations.
• Architectural parameters correlate with GPU-based token latency.

Method

LeakyLMs models token-generation timing on NVIDIA GPUs, characterizing latency with model configuration, then searches the architecture space to infer properties.

In practice

• Detect speculative decoding usage in remote LLMs.
• Estimate draft model context window (e.g., 128K tokens).
• Recover transformer layer count and hidden dimension size.

Topics

• LeakyLMs
• Timing Attacks
• LLM Architecture
• Inference Optimization
• Speculative Decoding
• NVIDIA GPUs

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, AI Security Engineer, Machine Learning Engineer

Related on AIssential

• Your LLM Is Guessing Ahead. Then It Checks Itself aka Speculative Decoding — Speculative decoding accelerates LLM inference by using a small model to guess tokens, verified by a large model in one pass.
• Security Risks in Language Models (LLMs) — LLMs face unique security threats from manipulation, data leakage, and supply chain vulnerabilities across their lifecycle.
• FloatDoor: Platform-Triggered Backdoors in LLMs — Platform-dependent numerical divergence in LLMs creates a novel attack surface for input-independent backdoors.
• SecureLLM: Using Inference-time Compositionality to Build Secure Language Models — SecureLLM secures LLMs by fine-tuning on segregated data and composing outputs at inference time based on user credentials.
• An Interpretable Latency Model for Speculative Decoding in LLM Serving — A new latency model explains speculative decoding performance in LLM serving under dynamic loads.
• Adversarial Prompts for Acceptance Collapse in Speculative Decoding — Speculative decoding's efficiency is vulnerable to prompt-based attacks that force draft rejection while preserving output quality.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.