Multi-Turn Tool Call with gpt-oss-chat

2026-03-23 · Source: DebuggerCafe · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, long

Summary

This article details the implementation of multi-turn tool calling capabilities within the `gpt-oss-chat` application, enabling a local AI assistant to autonomously utilize multiple tools like web search and local RAG (Retrieval Augmented Generation) within a single user turn. The system extends `gpt-oss-chat` to allow the assistant to make successive internal tool calls, such as searching user-uploaded documents and then the web, to synthesize comprehensive responses. The implementation involves minor logical changes in `api_call.py` and a modified system prompt in `utils/prompt.py`, which guides the assistant on when and how to use tools, including a `MAX_TOOL_CALLS` limit of 5 to prevent infinite loops. The `gpt-oss-20b` model is served using `llama.cpp` with a 32000 context length, demonstrating the workflow with a PDF document and web search APIs like Tavily and Perplexity.

Key takeaway

For AI Engineers developing local chat applications, implementing multi-turn tool calls significantly enhances assistant capabilities. You should focus on refining system prompts to guide tool selection and manage tool call limits to ensure efficient and relevant information retrieval. This approach allows your assistant to autonomously combine document retrieval and web search, providing more comprehensive answers to complex user queries.

Key insights

Local AI assistants can achieve multi-turn tool calling for complex queries by integrating web search and RAG.

Principles

• Prioritize accuracy using tools.
• Synthesize all tool results.
• Avoid duplicate tool calls.

Method

The method involves modifying the system prompt to guide tool usage, implementing a `MAX_TOOL_CALLS` limit, and iteratively calling tools based on stream events until a final response is generated or the limit is reached.

In practice

• Integrate `llama.cpp` for local model serving.
• Use Tavily or Perplexity for web search.
• Implement Qdrant for in-memory vector DB.

Topics

• Multi-Turn Tool Calling
• Local LLM Deployment
• Retrieval-Augmented Generation
• Web Search Integration
• System Prompt Engineering

Code references

• sovit-123/gpt-oss-chat

Best for: AI Engineer, Machine Learning Engineer, AI Chatbot Developer

Related on AIssential

• RAG Tool Call for gpt-oss-chat — Integrating RAG as a tool call allows LLMs to dynamically select relevant information sources, optimizing context and response.
• Evaluation: Multi-turn Conversations, Tool Use, Tracing, and Red Teaming — Multi-turn LLM evaluation requires assessing both individual turns and overall task completion using specialized metrics and frameworks.
• Web Search Tool with Streaming in gpt-oss-chat — LLMs can autonomously decide to use web search and generate queries, enhancing conversational AI.
• Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents — Multi-turn, tool-using agent interactions significantly degrade safety, necessitating proactive, self-exploring defenses.
• To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Calling — LLMs often misjudge when to use external tools, leading to suboptimal performance and inefficient resource use.
• App lets you put multiple LLMs into one group chat! Have them fight it out or collaborate. — Interacting LLMs in group chats can reveal novel insights and emergent conversational behaviors.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by DebuggerCafe.