Automatic Upgrades: best practice features for your lakehouse tables

2026-07-06 · Source: Databricks · Field: Technology & Digital — Data Science & Analytics, Cloud Computing & IT Infrastructure, Artificial Intelligence & Machine Learning · Depth: Intermediate, medium

Summary

Databricks has introduced Automatic Upgrades for Unity Catalog (UC) managed tables, a new capability designed to automatically apply best-practice features without manual effort. This system observes how existing tables are accessed over a 100-day window, verifies client compatibility with features like Row Tracking, Liquid Clustering, and Deletion Vectors, and then applies them via a background ALTER TABLE job. Once all existing tables in a schema are compatible, new tables created there inherit the features by default. Auto Upgrades ensures features are GA-only, have no material performance or cost regressions, and are reversible. Benefits include faster, more cost-efficient tables through optimizations like Parquet V2 and Column Mapping, enhanced open interoperability via Catalog Commits, and greater reliability under load with Checkpoint V2. Changes are visible in DESCRIBE HISTORY and a system.storage.table_auto_upgrade_operations_history system table.

Key takeaway

For Data Engineers or MLOps Engineers managing Databricks lakehouse environments, Auto Upgrades significantly reduces the operational burden of adopting new table features. You should prioritize converting your existing external tables to Unity Catalog managed tables to automatically gain performance, cost, and interoperability benefits. This ensures your data infrastructure evolves without manual ALTER TABLE marathons, allowing you to focus on higher-value tasks while maintaining full control over feature adoption.

Key insights

Automated table feature management enhances lakehouse performance, reliability, and interoperability by verifying client compatibility.

Principles

• Features must be GA and non-regressive.
• Comprehensive client compatibility is critical.
• User control via reversibility is essential.

Method

Auto Upgrades observes table access for 100 days, verifies all clients support a feature, then applies it via a background ALTER TABLE job. New tables inherit features from compatible schemas.

In practice

• Convert external tables to UC managed.
• Query system.storage.table_auto_upgrade_operations_history for audit.
• Disable specific features per table if needed.

Topics

• Unity Catalog
• Lakehouse Architecture
• Data Management Automation
• Table Features
• Databricks Runtime
• Data Governance

Best for: CTO, VP of Engineering/Data, AI Architect, Data Engineer, MLOps Engineer, Director of AI/ML

Related on AIssential

• Predictive Optimization at Scale: A Year of Innovation and What’s Next — Predictive Optimization automates data layout and maintenance for cost-effective, high-performance lakehouses.
• Introducing the Databricks Excel Add-in for Business Users — The Databricks Excel Add-in simplifies governed lakehouse data access for business users in Excel.
• Designing a Modern Enterprise Lakehouse with Databricks, DABs, and Medallion Architecture — Platform scalability in enterprise lakehouses relies on reusable engineering patterns and consistent deployment strategies, not just individual technologies.
• Full End-to-End Data Engineering Project in Databricks — Automating data ingestion and ETL pipelines in Databricks creates a robust, self-updating data engineering workflow.
• Databricks vs Snowflake: Choosing the Right Platform for Modern Data and AI Workloads — Databricks excels in AI/ML and data engineering, while Snowflake dominates SQL-based analytics and business intelligence.
• Databricks is no longer about tuning knobs — Databricks prioritizes abstraction and automation over granular control to serve less technical users and accelerate business value.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Databricks.