Data & AI Summit Takeaways: Breakout Sessions
Summary
The Data & AI Summit featured several breakout sessions highlighting practical data and AI applications. Scribd demonstrated a Databricks App for data quality and observability, which includes rule management, AI-driven rule suggestions, and historical trend analysis, building on the DQX framework. Mercedes Benz Korea showcased their approach to conversational data interaction, leveraging metric views and Genie to create a semantic layer for business decisions, underscoring the importance of consistent data modeling. Boeing presented a robust data contract implementation, moving beyond Wiki pages to an open-spec standard, AI-assisted authoring, automated data checks, and a subscription-based contract marketplace for enhanced trust at scale.
Key takeaway
For Data Engineers or MLOps Engineers building robust data platforms, consider integrating AI-driven solutions for data quality and governance. You should explore developing custom Databricks Apps for managing data quality rules and adopt open-spec data contracts with AI-assisted authoring to ensure trust at scale. This approach streamlines data reliability, automates checks, and fosters better coordination between data producers and consumers, reducing manual overhead and improving data integrity.
Key insights
Modern data platforms and AI enable scalable solutions for data quality, conversational analytics, and trust through robust data contracts.
Principles
- Data consistency is foundational for conversational AI.
- Open-spec standards enhance data contract coordination.
- AI can accelerate data quality rule creation and contract authoring.
Method
Implement data contracts using an open-spec standard, accelerate authoring with AI, compile specs into running checks, and establish a marketplace for stakeholder subscriptions.
In practice
- Develop custom data quality rules within Databricks Apps.
- Utilize metric views and Genie for conversational data experiences.
- Adopt AI-assisted data contract authoring for improved coordination.
Topics
- Data Quality
- Databricks Apps
- Data Contracts
- Conversational AI
- Semantic Layer
- AI-Assisted Authoring
Best for: CTO, VP of Engineering/Data, Director of AI/ML, Data Engineer, Data Scientist, MLOps Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Data Engineering on Medium.