Qwen3.5 Medium Models: Dense vs. MoE

2026-02-25 · Source: The Kaitchup – AI on a Budget · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Intermediate, quick

Summary

Alibaba's Qwen team has released three new "medium" models within the Qwen3.5 multimodal family: Qwen3.5-35B-A3B, Qwen3.5-122B-A10B, and Qwen3.5-27B, along with a base variant of Qwen3.5-35B-A3B designed for easier fine-tuning. These models incorporate Gated Deltanet, a linear attention mechanism, in 75% of their layers. This architectural choice aims to deliver high throughput and a small KV cache, which significantly reduces memory consumption, even when processing long context lengths. While currently too large for consumer GPUs in full precision, their design suggests they will become practical once aggressively quantized, with early experiments indicating strong robustness to low-bit quantization.

Key takeaway

For NLP Engineers evaluating new multimodal models for deployment, Qwen3.5's architectural choices, particularly Gated Deltanet, suggest strong potential for efficient inference. You should prioritize testing these models with aggressive low-bit quantization (e.g., 4-bit or 2-bit) to assess their performance and memory footprint on your target hardware, especially for applications requiring long context lengths on consumer-grade GPUs.

Key insights

Qwen3.5 medium models use Gated Deltanet for high throughput and low memory, showing robustness to quantization.

Principles

• Linear attention reduces KV cache size.
• Aggressive quantization can make large models practical.

In practice

• Consider Qwen3.5 for memory-constrained long context.
• Explore 4-bit or 2-bit quantization for deployment.

Topics

• Qwen3.5 Models
• Multimodal AI
• Linear Attention
• Model Quantization
• Memory Footprint

Best for: NLP Engineer, AI Engineer, Machine Learning Engineer, Deep Learning Engineer

Related on AIssential

• Qwen3.5: Scaling Hybrid Attention to 397B Parameters — Hybrid attention with Gated DeltaNet scales effectively, enabling cost-efficient, high-performing LLMs.
• Now in Foundry: Qwen3.5 Medium Model Series — Qwen3.5 models offer diverse VLM capabilities, from low-latency dense to cost-efficient MoE architectures.
• Qwen3.5 Quantization: Similar Accuracy, More Thinking — Best Models and Recipes — Quantization significantly reduces LLM memory footprint while often retaining substantial accuracy, but careful layer selection is crucial.
• Qwen3.6 (27B) Local Test | Better than Gemma 4? | Coding, OCR, Images with llama.cpp | 🔴 Live — Qwen 3.6 (27B) offers strong reasoning and multimodal capabilities but suffers from slow local inference speed.
• Alibaba's new open source Qwen3.5-Medium models offer Sonnet 4.5 performance on local computers — Alibaba's Qwen3.5 models offer high-performance, open-source LLMs with efficient local deployment and competitive API pricing.
• Qwen3.5-397B-A17B: the smallest Open-Opus class, very efficient model — Qwen3.5 offers multimodal, long-context capabilities with efficient sparse MoE architecture, enabling local deployment despite its large scale.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Kaitchup – AI on a Budget.