Ch 15 - People and Organizations: Data Modeling is a Full-Contact Sport
Summary
Chapter 15 of an upcoming book on Physical Data Modeling (PDM) emphasizes that organizational dynamics, rather than technical acumen, are the primary drivers of success or failure in data initiatives. Drawing parallels to "gym politics" and citing Tom DeMarco and Timothy Lister's "Peopleware," the chapter argues that data modeling is a sociological phenomenon often overlooked in technical resources. It illustrates this with the $7 billion failure of Target Canada's expansion, where misaligned incentives and organizational structures led to systemic data defects, and a "Customer 360" project that stalled due to departmental disagreements over customer definitions. The content highlights that data models are political artifacts reflecting influence and priorities, and that real-world constraints often necessitate compromises over theoretical ideals. It also introduces Conway's Law, explaining how organizational communication structures directly influence system and data model design, leading to "silo taxes" that hinder agility.
Key takeaway
For CTOs overseeing data initiatives, recognize that organizational alignment and communication structures are paramount. Your data architecture will inevitably reflect your organizational chart, so proactively design team structures (Inverse Conway Maneuver) to foster desired system outcomes. Prioritize securing executive consensus on core data definitions and assigning clear data ownership before embarking on complex technical modeling, as neglecting these "sociological" aspects will likely lead to project failure and wasted resources.
Key insights
Organizational dynamics and human interaction are more critical to data modeling success than technical expertise.
Principles
- Data models are political artifacts.
- Conway's Law dictates system design mirrors communication structure.
- People and organizational issues sink more IT projects than technical ones.
Method
Design the "social contract" around the data model by enforcing canonical units, setting explicit quality bars, and assigning named business owners with accountability for data quality.
In practice
- Secure executive alignment on definitions before technical work.
- Time-box discovery work in hierarchical organizations.
- Use data models to bridge organizational silos.
Topics
- Data Modeling
- Organizational Dynamics
- Conway's Law
- Data Silos
- Project Failure Analysis
Best for: CTO, Director of AI/ML, VP of Engineering/Data, Data Scientist
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Practical Data Modeling.