Enabling evolutionary database development: Part 3

2026-06-16 · Source: Thoughtworks Insights · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Advanced, extended

Summary

This article, published June 16, 2026, details scaling evolutionary database development for a fifty-developer team, including AI agents, using database branching with Lakebase. It identifies three critical structural elements for this scale: tier topology, which defines long-running branches for promotion paths; a robust permission model, designed once and automatically enforced; and the DBA's evolving role into a platform engineer. The approach replaces separate database instances with a single Lakebase parent and a hierarchy of logical branches, enabling benefits like unified pipelines, merge-based promotions, and reduced environment drift. It also outlines how AI agents operate within a structured Source Control Management (SCM) workflow, emphasizing guardrails, named refactorings, and human review to ensure maintainability and prevent issues like agents deleting tests.

Key takeaway

For MLOps Engineers or Database Administrators managing large development teams, you should transition to a branch-native database model using tools like Lakebase. This approach centralizes governance, automates environment provisioning, and integrates AI agents safely into your SCM workflows. By defining tier hierarchies and permission models upfront, you reduce toil, improve auditability, and enable faster, more consistent database changes across your organization.

Key insights

Evolutionary database development scales by treating environments as branches and enforcing governance via a platform.

Principles

• Tier topology uses long-running branches.
• Permission models are designed once, enforced automatically.
• DBA role shifts to platform architect.

Method

Implement a five-state SCM workflow (scaffold-complete, feature-claimed, pr-ready, ci-green, merged) with CLI-driven transitions and schema-validated state files, enforced by a common substrate.

In practice

• Define tier hierarchy and permission boundaries.
• Use Unity Catalog for policy inheritance.
• Implement audit trail capture for all actions.

Topics

• Database Branching
• Evolutionary Database Development
• Lakebase
• AI Agents
• SCM Workflows
• Unity Catalog
• Platform Engineering

Code references

• databricks-solutions/lakebase-app-dev-kit
• databricks-solutions/lakebase-scm-extension

Best for: Software Engineer, Data Engineer, MLOps Engineer

Related on AIssential

• Enabling Evolutionary Database Development: Database branching with Lakebase, the conclusion — Copy-on-write database branching in 2026 enables scalable, isolated database development for human and agent teams.
• How agentic software development will change databases — AI agents are transforming software development, demanding databases with evolutionary branching, zero-cost elasticity, and open ecosystems.
• H2O MLOps Enterprise Model Registry & Hugging Face | Part 8 Integration | Part 8 — A robust model repository is essential for managing and governing a large, diverse portfolio of AI models in production.
• The Join That Kills Your Model: How Cardinality Explosions in Production Databases Destroy ML… — Database schema changes can cause silent cardinality explosions, destroying ML feature integrity.
• Escaping the SQL Jungle — Unmanaged ELT transformations lead to a "SQL jungle"; a structured transformation layer restores order and consistency.
• Trustworthy Self-Composable Big-Data-as-a-Service: An LLM-Orchestrated Multi-Agent Framework for Automated Data Engineering, AutoML, MLOps Deployment, and Drift-Aware Lifecycle Optimization — LLM-orchestrated multi-agent systems enable trustworthy, adaptive, and production-oriented Big-Data-as-a-Service lifecycle automation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Thoughtworks Insights.