Enabling Evolutionary Database Development: database branching with Lakebase, continued

2026-06-05 · Source: Databricks · Field: Technology & Digital — Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Advanced, long

Summary

Databricks Lakebase introduces copy-on-write database branching, fundamentally changing Evolutionary Database Development by making a one-second, zero-storage-at-creation branch of a terabyte-scale production database an O(1) operation. This capability lifts the constraint that previously limited seven key practices from a 2003 essay, such as "everybody gets their own database instance." The article details how Lakebase's decoupled architecture, separating compute and storage, enables per-developer, per-PR, and per-experiment database branches. This shift allows for asynchronous DBA collaboration, continuous integration of database changes against isolated environments, and the practical application of database refactorings. It also introduces four new practices, including destructive testing and A/B variant prototyping at the database level, bringing the total to eleven practices for 2026. The series illustrates these changes through the character Jen's workflow across three scopes.

Key takeaway

For DevOps Engineers and Software Engineers managing database changes, Databricks Lakebase's copy-on-write branching fundamentally alters your CI/CD strategy. You can now automate per-PR database instances, eliminating shared development database contention. This enables robust, isolated testing against production-shaped data. Integrate database refactorings and destructive testing directly into your daily workflow, significantly improving development velocity and reducing deployment risks. Consider adopting Lakebase to streamline your evolutionary database development practices.

Key insights

Copy-on-write database branching makes per-instance database development feasible, removing key constraints for evolutionary design.

Principles

• Database branching enables per-PR isolation.
• DBA review shifts to asynchronous design focus.
• Migrations must be idempotent for reliability.

Method

Implement CI workflows using GitHub Actions to automatically create Lakebase branches for each pull request, apply migrations, run tests, post schema diffs, and destroy branches on merge.

In practice

• Use "databricks postgres create-branch" for dev instances.
• Fork production for A/B variant prototyping.
• Run destructive tests on ephemeral branches.

Topics

• Evolutionary Database Design
• Databricks Lakebase
• Database Branching
• CI/CD Pipelines
• Database Refactoring
• DevOps Workflows
• Data Governance

Code references

• databricks-solutions/lakebase-app-dev-kit

Best for: Software Engineer, Data Engineer, DevOps Engineer

Related on AIssential

• Enabling Evolutionary Database Development: database branching with Lakebase — Copy-on-write database branching eliminates shared database bottlenecks, enabling true per-developer isolation and faster feedback.
• Nextdoor’s Database Evolution: A Scaling Ladder — Nextdoor's database architecture evolved from a single PostgreSQL instance to a sophisticated distributed system to support millions of users, addressing scaling challenges through a disciplined progr…
• Backstage with Lakebase, part 2 — Unified governance for operational databases simplifies security, compliance, and cost attribution while enabling rapid development workflows.
• The Join That Kills Your Model: How Cardinality Explosions in Production Databases Destroy ML… — Database schema changes can cause silent cardinality explosions, destroying ML feature integrity.
• Auto Loader: End-to-End Schema Evolution — Auto Loader provides scalable, incremental data ingestion with configurable schema evolution for cloud storage.
• Branches, Diffs, and SQL: How Dolt Powers Agentic Workflows — Dolt provides Git-style version control for SQL databases, enabling row-level branching, merging, and diffing for data and schema.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Databricks.