The Next Era of the Open Lakehouse: Apache Iceberg™ v3 in Public Preview on Databricks

2026-04-09 · Source: Databricks · Field: Technology & Digital — Data Science & Analytics, Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Intermediate, medium

Summary

Databricks has launched Iceberg v3 support in Public Preview, enhancing its open lakehouse capabilities. This update introduces Row Lineage, Deletion Vectors, and the VARIANT type, addressing challenges in incremental data processing and semi-structured data analysis. Row Lineage assigns a permanent row ID and sequence number to each row, enabling efficient identification of changes. Deletion Vectors improve data manipulation performance by up to 10x by logically deleting rows without immediate file rewrites. The VARIANT type allows storing semi-structured data directly within Iceberg tables, simplifying ingestion and querying with standard SQL. Unity Catalog further unifies governance and interoperability across various catalogs and engines, supporting fine-grained access control and seamless integration between Delta Lake and Iceberg ecosystems.

Key takeaway

For CTOs and VPs of Data evaluating lakehouse strategies, Iceberg v3 on Databricks offers significant advancements in data processing efficiency and flexibility for semi-structured data. Your teams can reduce operational overhead and costs by leveraging native CDC capabilities and simplified handling of evolving data schemas, while Unity Catalog ensures unified governance across diverse data environments.

Key insights

Iceberg v3 enhances open lakehouse capabilities with features for incremental processing and semi-structured data.

Principles

• Incremental processing reduces costs.
• Semi-structured data needs flexible schemas.
• Unified governance improves data security.

Method

Iceberg v3 uses Row Lineage for change identification and Deletion Vectors for 10x faster logical row deletion. The VARIANT type stores semi-structured data directly, allowing SQL querying without ETL.

In practice

• Use Row Lineage for CDC pipelines.
• Employ Deletion Vectors for faster updates.
• Store logs with VARIANT for direct SQL access.

Topics

• Apache Iceberg v3
• Databricks Unity Catalog
• Row Lineage
• Deletion Vectors
• VARIANT Data Type

Best for: CTO, VP of Engineering/Data, Director of AI/ML, Data Engineer, MLOps Engineer, AI Architect

Related on AIssential

• Advancing Apache Iceberg on Databricks: Iceberg v3 GA, Open Sharing, and Unified Governance — Unity Catalog unifies Apache Iceberg governance, sharing, and performance across diverse engines and external catalogs.
• Iceberg is harder than you think — Iceberg implementation is complex, requiring careful data transport, catalog integration, and engine optimization.
• Open Platform, Unified Pipelines: Why dbt on Databricks is Accelerating — Consolidating dbt workflows on an open lakehouse platform enhances data governance, performance, and operational simplicity.
• Your Data, Your Lake: How Observe Uses Iceberg and Streaming ETL for Observability — Lakehouse architectures can provide scalable, cost-efficient observability by centralizing diverse telemetry data.
• Data Lake vs Data Warehouse vs Lakehouse vs Data Mesh: What’s the Difference? — Data architectures evolve from structured reporting to flexible raw storage, then to unified platforms and decentralized organizational models.
• Why Databricks is Winning the Data Wars in 2026 — A Technical Breakdown — Databricks' Lakehouse architecture unifies data warehousing and data lake functionalities on open standards, driven by governance and AI needs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Databricks.