Tool support

2024-07-24 · Source: Ollama Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, quick

Summary

Ollama announced on July 25, 2024, that its platform now supports tool calling for popular large language models, including Llama 3.1, Mistral Nemo, Firefunction v2, and Command-R +. This new capability allows models to interact with external functions, APIs, web browsers, and code interpreters to perform more complex tasks. Users can enable tool calling by providing a list of available tools via the `tools` field in Ollama's API, which then returns a `tool_calls` response. The feature is also compatible with OpenAI's endpoint, enabling seamless integration for developers already using that API. Ollama plans future improvements such as streaming tool calls and forced tool choice.

Key takeaway

For AI Engineers building applications with local LLMs, Ollama's new tool calling support means you can significantly expand model capabilities beyond text generation. You should explore integrating functions, APIs, or web browsing tools to enable your models to perform more dynamic and complex real-world tasks, potentially reducing reliance on external API services for certain models.

Key insights

Ollama now enables LLMs to interact with external tools for complex tasks via a new API feature.

Principles

• Models can perform complex tasks by integrating external tools.
• Standardized API fields simplify tool integration for LLMs.

Method

Provide a list of available tools in the `tools` field of Ollama's chat API request. The model responds with `tool_calls` for execution, and tool outputs are fed back via messages with the `tool` role.

In practice

• Integrate web browsing capabilities into LLM applications.
• Use code interpreters for dynamic problem-solving.
• Connect LLMs to custom APIs for real-world interactions.

Topics

• Tool Calling
• Ollama API
• Llama 3.1
• OpenAI Compatibility
• Large Language Models

Code references

• user-attachments/assets
• ollama/ollama
• ollama/ollama-python
• ollama/ollama-js

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• Ollama: Unleash the Power of Local LLMs on Your Machine — Ollama enables private, cost-effective, and low-latency local LLM inference and customization via a Python library.
• Web search — Ollama's new web search API enhances LLMs by providing real-time web data to improve accuracy and reduce hallucinations.
• Llama 3 — Llama 3 offers substantial improvements over Llama 2, now locally runnable via Ollama.
• From Local LLM to Tool-Using Agent — Local LLMs can become powerful tool-using agents by integrating with agent runtimes and external services.
• Best Ollama Alternatives in 2026: Top Local AI Tools for Developers & Teams — Local AI tools offer enhanced control, privacy, and performance over cloud-based solutions for evolving AI workflows.
• 5 Cool Things I Did with Local Language Models — Local LLMs offer superior privacy, cost, and offline capabilities for sensitive data and proprietary workflows.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Ollama Blog.