Data Strategy: a map for choosing the right architecture

· Source: Data Engineering on Medium · Field: Technology & Digital — Data Science & Analytics, Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Intermediate, medium

Summary

This article presents a data strategy framework, likening it to a water system, to guide the selection of appropriate data architectures. It outlines seven critical dimensions: Meaning (metrics, MDM), Foundation (storage like data warehouse, lake, lakehouse), Time (update frequency: batch, CDC, streaming), Ownership (centralized, federated, data mesh), Connection (data fabric, catalogs, APIs), Consumption (reports, AI), and Control (governance, quality, security). The piece then details various architectural approaches, including data warehouses for stable needs, data lakes for uncertain uses, and data lakehouses for integrated BI/AI. It also covers organizational models like data mesh for distributed ownership and technologies such as data fabric for fragmented environments, emphasizing that architecture is a combination of these dimensions.

Key takeaway

For AI Architects or Data Engineers designing new data platforms, carefully evaluate your organization's specific needs across the seven data dimensions before selecting technologies. If you face scattered data, consider data fabric or catalogs for integration; for urgent operational decisions, explore ODS or streaming. Prioritize defining data meaning and ownership with MDM and governance to prevent quantification bias and ensure your architecture truly supports business outcomes.

Key insights

Data architecture selection must align with business needs, data meaning, and organizational context.

Principles

Method

Assess data needs across seven dimensions: meaning, foundation, time, ownership, connection, consumption, and control, then match to suitable architectures like data lakehouse or data mesh.

In practice

Topics

Best for: Data Engineer, Data Scientist, AI Architect

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Data Engineering on Medium.