5 Small Language Models for Agentic Tool Calling

2025-07-08 · Source: KDnuggets · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Advanced, medium

Summary

Five small language models (SLMs) are highlighted for their compact, open-weight design and robust support for structured tool calling, crucial for agentic AI systems. These models include SmolLM3-3B (3B parameters, released July 8, 2025, by Hugging Face), Qwen3-4B-Instruct-2507 (4B parameters, August 6, 2025, by Alibaba), Phi-3-mini-4k-instruct (3.8B parameters, April 2024, by Microsoft), Gemma-4-E2B-it (2.3B effective parameters, April 2, 2026, by Google DeepMind), and Mistral-7B-Instruct-v0.3 (7.25B parameters, May 27, 2024, by Mistral AI). Each model offers distinct features such as dual-mode reasoning, multilingual support, long context windows, multimodal capabilities, and specific architectural innovations like Grouped Query Attention (GQA) or Per-Layer Embeddings (PLE), making them suitable for various constrained hardware and application scenarios.

Key takeaway

For AI Architects and NLP Engineers evaluating models for agentic systems, consider these compact, open-weight SLMs to reduce costs and latency compared to larger frontier models. Assess your specific needs for context length, multilingual support, multimodal input, or licensing to select the optimal model. For example, Gemma-4-E2B-it is ideal for multimodal edge deployments, while Mistral-7B-Instruct-v0.3 offers strong general instruction-following.

Key insights

Small, open-weight language models now offer robust tool-calling for agentic AI, reducing reliance on large frontier models.

Principles

• Agentic AI requires reliable tool calling.
• Small models can rival large models in capability.
• Open-weight models foster innovation and deployment.

Method

These SLMs enable agentic workflows by supporting structured tool calling via JSON/XML, Python-style function calls, native function calling, or dedicated chat template tokens.

In practice

• Deploy SLMs on edge devices for low-latency agents.
• Utilize Qwen-Agent for simplified tool-calling integration.
• Consider Phi-3-mini for on-device reasoning tasks.

Topics

• Agentic Tool Calling
• Open-weight Models
• SmolLM3-3B
• Qwen3-4B-Instruct
• Phi-3-mini

Best for: AI Architect, NLP Engineer, AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• When to use Small LM for AI Agents: New Insights — Small, open-weight LLMs can handle many AI agent tasks, significantly reducing operational costs.
• Best Small Language Models on Hugging Face Right Now! — Small language models now rival larger models in performance, enabling local, cost-effective deployment.
• Models - from cloud to edge | Mistral - Mistral AI — Mistral AI provides a comprehensive suite of advanced deployable models, from general-purpose to specialized, covering text, code, vision, and voice.
• Rethinking Scale: Deployment Trade-offs of Small Language Models under Agent Paradigms — Agent paradigms, especially single-agent tool use, can significantly enhance Small Language Model capabilities.
• not much happened today — The AI landscape is rapidly evolving with open-weight models nearing frontier capabilities, while agentic systems and specialized benchmarks drive practical application and efficiency.
• [AINews] Gemma 4: The best small Multimodal Open Models, dramatically better than Gemma 3 in every way — Gemma 4 advances open-source AI with multimodal capabilities, efficient architecture, and a permissive Apache 2.0 license.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by KDnuggets.