Inkling, Gemma 4 Updates, and 1-Bit Qwen3.6

2026-04-15 · Source: The Kaitchup – AI on a Budget · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Advanced, medium

Summary

Thinking Machines has released Inkling, a 975-billion-parameter multimodal model with 41 billion active parameters per token, contributing to global open-weight AI competition. Inkling features learned relative positional representations for long-context modeling, a global-local attention pattern, short convolutions, and a mixture-of-experts design with 256 routed and two shared experts. It also employs parameter-specific optimization and learns to adapt reasoning to a compute budget. Concurrently, Gemma 4 received runtime updates, including broader FlashAttention 4 support for NVIDIA Hopper GPUs, visual-token budgets for image processing, and fixes to chat/tool-use templates, improving agent reliability. PrismML's Bonsai Qwen3.6 introduces binary (3.9GB) and ternary (5.9GB) quantized versions of Qwen3.6-27B, applying low-bit formats across most of its language network and requiring custom CUDA/MLX kernels for efficient inference, demonstrating significant compression with usable accuracy.

Key takeaway

For MLOps engineers focused on optimizing large language model deployments, these updates highlight critical avenues for efficiency. Gemma 4's enhanced FlashAttention 4 support and visual-token budgets offer direct controls to improve throughput and manage context for multimodal agents. Furthermore, Bonsai Qwen3.6's extreme binary/ternary quantization demonstrates viable pathways for deploying powerful models like Qwen3.6-27B into memory-constrained environments, such as high-end mobile applications. Evaluate these advancements to significantly reduce memory footprint and accelerate inference for your specific agentic workflows or edge computing needs.

Key insights

Recent advancements focus on optimizing large language models for efficiency, context, and tool integration.

Principles

• Maintaining global competition in open-weight AI fosters innovation.
• Parameter-specific optimization can enhance training efficiency.
• Models can learn to adapt reasoning to compute budgets.

Method

Inkling's long-context architecture combines learned relative positional representations with a 5:1 sliding-window to global-attention layer ratio and short convolutions. Bonsai uses custom CUDA/MLX kernels for direct operation on packed binary/ternary weights.

In practice

• Utilize Gemma 4's visual-token budgets for fine-grained image processing control.
• Apply parameter-specific optimizers like Muon/Adam based on parameter structure.
• Explore binary/ternary quantization for extreme model compression on edge devices.

Topics

• Open-Weight AI
• Multimodal LLMs
• Model Quantization
• Long-Context Attention
• Inference Optimization
• Gemma 4

Best for: AI Engineer, NLP Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Gemma 4 Technical Report — Gemma 4 advances open-weight multimodal LLMs with architectural innovations for efficiency, reasoning, and performance across diverse modalities.
• Qwen3.5-397B-A17B: the smallest Open-Opus class, very efficient model — Qwen3.5 offers multimodal, long-context capabilities with efficient sparse MoE architecture, enabling local deployment despite its large scale.
• Qwen3.6 Local Test | Can it Beat Gemma 4? | Coding, OCR, Image Understanding with llama.cpp | 🔴 Live — Qwen 3.6 offers strong agentic coding and visual understanding, but its verbose reasoning can impact efficiency.
• Nanbeige4.1: Only 3B Parameters, but as Good as Qwen3 32B? — Advanced post-training and hybrid attention mechanisms enable compact and massive LLMs to achieve impressive performance.
• Best Small Language Models on Hugging Face Right Now! — Small language models now rival larger models in performance, enabling local, cost-effective deployment.
• Gemma 4 is not your standard transformer — Gemma 4's architectural deviations suggest a re-evaluation of core transformer assumptions, particularly regarding token information flow.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Kaitchup – AI on a Budget.