AiChemy: Next-Generation Agent with MCP, Skills and Custom Data for Drug Discovery

2026-04-03 · Source: Databricks · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Health & Medical Research · Depth: Intermediate, short

Summary

Databricks offers a framework for building custom multi-agent supervisors by integrating public MCP (Multi-Cloud Platform) servers with proprietary data. This system utilizes five workers: OpenTargets, PubMed, PubChem for external knowledge graphs and literature, and proprietary Drug Library (Genie) for structured drug properties with text-to-SQL, alongside a Chemical Library (Vector Search) for unstructured chemical data and similarity search. Users can connect to public MCP servers via Unity Catalog connections, transform structured tables into Genie spaces, and create vector indexes for unstructured data. The multi-agent supervisor can be assembled using either a no-code "Agent Bricks" UI or advanced Databricks Notebooks for Langgraph supervisors, enabling deployment as an MLflow AgentServer with a React UI. All agent invocations are automatically logged and traced to MLflow experiments for evaluation and monitoring.

Key takeaway

For AI Engineers developing multi-agent systems, consider Databricks' approach to integrate diverse data sources. You can rapidly prototype with no-code Agent Bricks or use Notebooks for advanced features like agentic memory. Ensure robust monitoring by leveraging MLflow tracing for end-to-end observability, which is crucial for debugging and optimizing agent performance in production environments.

Key insights

Databricks enables building custom multi-agent systems by integrating public and proprietary data sources.

Principles

• Combine external and internal data for comprehensive agents
• Utilize vector search for unstructured data retrieval
• Employ text-to-SQL for structured data querying

Method

Prepare components (MCP connections, Genie spaces, vector indexes), then build the supervisor using either no-code Agent Bricks or Databricks Notebooks for advanced features, and finally deploy and monitor via MLflow.

In practice

• Connect to external APIs via Unity Catalog
• Convert structured data to Genie spaces for SQL access
• Create vector indexes for similarity search

Topics

• Multi-Agent Systems
• Drug Discovery
• Databricks Platform
• MCP Servers
• Vector Search

Code references

• databricks-industry-solutions/aichemy

Best for: AI Engineer, MLOps Engineer, Research Scientist

Related on AIssential

• Accelerating pharmaceutical research with a multi-agent AI system — Multi-agent AI systems can transform fragmented data into actionable insights, significantly accelerating complex research processes.
• Custom Agents now available on Databricks — Databricks Custom Agents streamline AI agent development and deployment with serverless infrastructure and integrated governance.
• Building Real-Time Product Search on Databricks — Modern product search requires an integrated platform for real-time ingestion, retrieval, and refinement, balancing speed with relevance and business outcomes.
• MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery — MLEvolve enables LLM agents to self-evolve for automated machine learning algorithm discovery by integrating advanced search and memory.
• An AI agent coding skeptic tries AI agent coding, in excessive detail — Modern AI agents, guided by explicit `AGENTS.md` files, can generate highly optimized, performant code across multiple languages.
• Learn IDP in Databricks in Under 2 Hours! — Databricks IDP leverages AI functions for flexible, accurate extraction and classification of unstructured document data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Databricks.