Introducing Cross-Engine ABAC

2026-06-02 · Source: Databricks · Field: Technology & Digital — Data Science & Analytics, Cybersecurity & Data Privacy, Cloud Computing & IT Infrastructure · Depth: Intermediate, short

Summary

Databricks has announced the Beta of cross-engine ABAC, extending its lakehouse vision for open storage, open access, and unified governance. This new capability enables enterprises to enforce attribute-based access controls (ABAC) defined in Unity Catalog across external engines using Iceberg REST Catalog APIs. Unity Catalog becomes the first catalog to deliver this cross-engine ABAC enforcement, applying tag-based row filters and column masks universally. This eliminates the previous tradeoff where security teams had to duplicate policies across engines or risk over-granting access. Cross-engine ABAC supports the full expressiveness of Unity Catalog policies, including tag-based rules and SQL UDFs, and works with any Iceberg REST client. Currently, it supports Apache Spark via Iceberg-Spark and Delta-Spark connectors, with Starburst and DuckDB integrations planned. The system functions by having Unity Catalog evaluate user entitlements and policies, returning a filtered scan plan to the engine, which then processes only authorized data.

Key takeaway

For MLOps Engineers or Data Architects managing data governance across a multi-engine lakehouse, you should evaluate Databricks' new cross-engine ABAC Beta. This capability allows you to define fine-grained access policies once in Unity Catalog, ensuring consistent enforcement of row filters and column masks across Apache Spark and other Iceberg REST clients. This significantly reduces policy duplication and security risks associated with siloed governance, streamlining your data access management. Consider enabling the preview and testing its integration with your existing Iceberg-Spark or Delta-Spark setups.

Key insights

Unity Catalog's cross-engine ABAC centralizes fine-grained data access policy enforcement across diverse data engines via Iceberg REST Catalog APIs.

Principles

• Unified governance simplifies security.
• Open standards enable broad interoperability.
• Centralized enforcement secures untrusted engines.

Method

External engines send scan requests to Unity Catalog. Unity Catalog evaluates policies, then returns a filtered scan plan, ensuring the engine only processes authorized data.

In practice

• Enable "Cross-engine ABAC" preview.
• Define tag-based row/column policies.
• Query via Iceberg-Spark connector.

Topics

• Unity Catalog
• Attribute-Based Access Control
• Iceberg REST Catalog
• Data Governance
• Lakehouse Architecture
• Apache Spark

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Architect, MLOps Engineer, Data Engineer

Related on AIssential

• Advancing Apache Iceberg on Databricks: Iceberg v3 GA, Open Sharing, and Unified Governance — Unity Catalog unifies Apache Iceberg governance, sharing, and performance across diverse engines and external catalogs.
• How Delta Sharing Supports ABAC Sharing for Providers and Recipients — ABAC-enabled Delta Sharing allows secure, governed data exchange with independent recipient-side policy enforcement.
• Designing a Modern Enterprise Lakehouse with Databricks, DABs, and Medallion Architecture — Platform scalability in enterprise lakehouses relies on reusable engineering patterns and consistent deployment strategies, not just individual technologies.
• Fine-tune LLM with Databricks Unity Catalog and Amazon SageMaker AI — Integrate Databricks Unity Catalog with AWS ML services for governed LLM fine-tuning and end-to-end data lineage.
• From siloed data to unified insights: Cross-account Athena Access for Amazon Quick — Cross-account Athena access unifies BI by enabling secure, cost-attributed querying of distributed data via IAM role chaining.
• Why Databricks is Winning the Data Wars in 2026 — A Technical Breakdown — Databricks' Lakehouse architecture unifies data warehousing and data lake functionalities on open standards, driven by governance and AI needs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Databricks.