Testing watsonx Orchestrate Agents with Bob

2026-06-16 · Source: Niklas Heidloff · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, medium

Summary

IBM Bob, an AI-based software development tool, can be extended to test watsonx Orchestrate agents, specifically enabling multi-turn conversations. This post details how to enhance an Orchestrate skill to invoke REST APIs, overcoming the "orchestrate" CLI's limitation in handling multi-turn interactions. The described method is optimized for the local Orchestrate Developer Environment and uses ADK version 2.0.10. An example flow illustrates creating a "Trip Booking" agent, installing the "orchestrate" CLI, generating agent code, importing resources, and then executing both single-turn and multi-turn conversation tests. These tests utilize a "wxo-chat.sh" script that programmatically interacts with watsonx Orchestrate agents via its REST API, demonstrating a complete agent development and verification lifecycle.

Key takeaway

For AI Engineers developing watsonx Orchestrate agents, you should integrate REST API calls into your testing workflow to enable robust multi-turn conversation validation. This approach, demonstrated with IBM Bob and ADK version 2.0.10, ensures comprehensive agent verification beyond single-turn interactions. Implement a "wxo-chat.sh"-like script to programmatically manage conversation threads, ensuring your agents are thoroughly tested before deployment. This is crucial for agents requiring complex, stateful interactions.

Key insights

Extending IBM Bob with watsonx Orchestrate REST APIs enables robust multi-turn conversation testing for AI agents.

Principles

• "Deployed ≠ verified."
• Derive tests from agent's own spec.
• Establish dependencies before references.

Method

Extend an Orchestrate skill to call REST APIs for multi-turn conversation testing. The process involves activating the skill, installing the "orchestrate" CLI, generating agent code, importing resources, then executing single- and multi-turn tests via a "wxo-chat.sh" script.

In practice

• Use "wxo-chat.sh" for programmatic agent chat.
• Test agents with single-turn conversations.
• Test agents with multi-turn conversations.

Topics

• IBM Bob
• watsonx Orchestrate
• AI Agents
• Multi-turn Conversations
• REST API Testing
• Agent Development Kit

Code references

• nheidloff/orchestrate-bob

Best for: AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Watsonx Orchestrate Debug Skill for IBM Bob — IBM Bob's custom debug skill automates local watsonx Orchestrate agent issue identification and resolution.
• Simulate realistic users to evaluate multi-turn AI agents in Strands Evals — ActorSimulator enables scalable, realistic multi-turn AI agent evaluation through structured, goal-driven user simulation.
• Enterprise AI Finds its Voice: ElevenLabs and IBM Bring Premium Voice Capabilities to Agentic AI — Integrating premium voice AI into agentic platforms enhances user trust and meets enterprise security and scalability demands.
• Leveraging gpt-oss-120b in watsonx Orchestrate — Strong LLMs like gpt-oss-120b enable autonomous orchestrator agents in multi-agent systems.
• Conversational ML Workflows via the H2O AI Super Agent in Enterprise h2oGPTe | Part 21 — The h2O Super Agent enables autonomous system orchestration via natural language, acting as an IDE.
• How to perform a structured evaluation of AI conversational solutions — Structured, multi-dimensional evaluation is crucial for complex RAG-based AI systems to diagnose and improve performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Niklas Heidloff.