Amplitude-Only FFN Intervention for Tool-Structured LLM Inference Method: Gated Evaluation Protocol, and Cross-Model Empirical Results

2026-07-13 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

Amplitude Gating (AG) is a novel inference-time feed-forward network (FFN) intervention method designed to improve structured outputs in large language models without retraining weights. Developed as a non-destructive alternative to Orthogonal Residual Projection (ORP), AG modulates only activation magnitudes, preserving pretrained FFN weight directions. The method defines a fine-grained intervention system across P1/P2/P3 and branch-specific sites. Researchers introduced a gated evaluation protocol that separates combination-oracle headroom from fixed configurations and learned gates, using task-aware metrics. Empirical results across Qwen3.5-9B, Qwen3-8B, and Qwen2.5-7B show AG is strongest on tool-structured tasks. On Qwen3.5-9B, a category-level learned gate boosted tool/structured/agentic performance from 38.66% to 42.92% (+4.27 percentage points), with Hermes function-call tasks improving by about +7.6 points. Qwen3-8B saw Hermes JSON mode improve by +11.36 points.

Key takeaway

For Machine Learning Engineers optimizing LLM inference for structured outputs, consider Amplitude Gating (AG) as a promising technique. Your teams can achieve significant performance improvements, particularly on tool-structured tasks and function-call generation, without costly model retraining. Focus on model- and category-specific routing for deployment, as results vary across models like Qwen3.5-9B and Qwen3-8B. Further online validation and broader cross-model evaluation are necessary to confirm widespread applicability.

Key insights

Amplitude Gating (AG) improves LLM structured outputs by modulating FFN activation magnitudes during inference without retraining.

Principles

• FFN intervention can enhance structured LLM outputs.
• Preserving FFN weight directions is crucial for non-destructive intervention.
• Model- and category-specific routing is needed for deployment.

Method

AG modulates FFN activation magnitudes at P1/P2/P3 and branch-specific sites during LLM inference to improve structured output quality.

In practice

• Apply AG to Qwen3.5-9B for tool/structured/agentic tasks.
• Target Hermes function-call tasks for significant gains.
• Implement category-level learned gates for specific models.

Topics

• Large Language Models
• FFN Intervention
• Amplitude Gating
• Tool-Structured Inference
• Qwen Models
• Function Calling

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation — Soft-prompt tuning efficiently reveals true LLM knowledge by overcoming format-following limitations in benchmarks.
• BaseRT: Advancing Best-in-Class LLM Inference with Apple M5 Neural Accelerators — BaseRT utilizes Apple M5 Neural Accelerators via Metal 4 tensor API for superior LLM inference performance.
• Qwen3.5: Scaling Hybrid Attention to 397B Parameters — Hybrid attention with Gated DeltaNet scales effectively, enabling cost-efficient, high-performing LLMs.
• Why Scale Will Not Solve AGI | Vishal Misra - The a16z Show — Vishal Misra's research models Large Language Models (LLMs) as a giant, sparse matrix representing token probability distributions, explaining how they perform in-context learning through real-time Bayesian updating.
• Dots.llm1.inst: The Instruction-Tuned Mixture-of-Experts Language Model You Need to Know About — `dots.llm1.inst` is an efficient MoE LLM offering Qwen2.5-72B-level performance with 14B active parameters for multilingual instruction-following.
• GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Models — GRASPrune enables efficient LLM serving by structured pruning with a global budget and post-pruning calibration.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.