Expanded interoperability with Unity Catalog Open APIs

2026-05-14 · Source: Databricks · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Cloud Computing & IT Infrastructure · Depth: Intermediate, short

Summary

Unity Catalog (UC) has expanded its open lakehouse capabilities with new features for external access to managed tables and credential vending, now in Beta and Generally Available (GA) respectively. UC managed tables, which utilize Predictive Optimization and Liquid Clustering for up to 20x faster queries and 50% lower storage costs, can now be created, read, and written to by external engines like Apache Spark, Flink, and DuckDB with full transactional safety and streaming support. This is enabled by credential vending, a GA feature that issues short-lived, scoped credentials to external engines on demand, supporting machine-to-machine (M2M) OAuth and automatic token refresh. Volume credential vending is also in Public Preview, extending this secure access to unstructured data. Delta Kernel, an open-source library, simplifies integration for connector developers, fostering a growing ecosystem.

Key takeaway

For data platform teams seeking to unify data governance across diverse compute engines, Unity Catalog's new external access to managed tables and credential vending capabilities are critical. You can now enable external engines to securely create, read, and write to optimized Delta tables and volumes, ensuring centralized control and auditability without data duplication. Consider enrolling in the Beta and implementing credential vending to enhance security and interoperability across your data and AI platforms.

Key insights

Unity Catalog now offers secure, optimized external access to managed tables and unstructured data via credential vending.

Principles

• Centralized governance enhances data consistency.
• Open APIs prevent vendor lock-in.
• Short-lived credentials improve security posture.

Method

Unity Catalog issues short-lived, scoped credentials via credential vending, allowing external engines to authenticate and access managed tables and volumes, with policies enforced centrally and M2M OAuth support.

In practice

• Use Delta-Spark 4.2 with Unity Catalog 0.4.1.
• Enable external data access on your metastore.
• Convert existing external tables to managed tables.

Topics

• Unity Catalog
• Open APIs
• Managed Tables
• Credential Vending
• Delta Kernel

Code references

• delta-io/delta
• unitycatalog/unitycatalog

Best for: CTO, VP of Engineering/Data, Director of AI/ML, Data Engineer, MLOps Engineer, AI Architect

Related on AIssential

• Backstage with Lakebase, part 2 — Unified governance for operational databases simplifies security, compliance, and cost attribution while enabling rapid development workflows.
• Unity AI Gateway: How to Connect Agents to External MCPs Securely — Unity AI Gateway simplifies secure agent access to external tools by centralizing authentication and governance.
• Azure for Data Engineers Part 3: Virtual Machines, SQL Database, Key Vault, Event Hubs, and Stream… — Integrating Azure services enables scalable, secure, real-time data pipelines from ingestion to structured analytics.
• Fine-tune LLM with Databricks Unity Catalog and Amazon SageMaker AI — Integrate Databricks Unity Catalog with AWS ML services for governed LLM fine-tuning and end-to-end data lineage.
• Frontier Experiences — Notes from the Databricks Data + AI Summit 2026 in San Francisco — The Databricks Data + AI Summit 2026 emphasized a governance-first, integrated architecture for enterprise AI, moving beyond fragmented experiments to production-ready agentic systems.
• Designing a Modern Enterprise Lakehouse with Databricks, DABs, and Medallion Architecture — Platform scalability in enterprise lakehouses relies on reusable engineering patterns and consistent deployment strategies, not just individual technologies.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Databricks.