Holiday Special: Joe Reis - A Journey around the World of Data (Eng)

«Data Management is an interesting one: If it fails, what’s the feedback loop?»

For the Holiday Special of Season 4, we’ve invited the author of «Fundamentals of Data Engineering», podcast host of the «Joe Reis Show», «Mixed Model Arts» sensei, and «recovering Data Scientist» Joe Reis.
Joe has been a transformative voice in the field of data engineering and beyond.
He is also the author of the upcoming book with the working title "Mixed Model Arts", which redefines data modeling for the modern era. 

This episode covers the evolution of data science, its early promise, and its current challenges. Joe reflects on how the role of the data scientist has been misunderstood and diluted, emphasizing the importance of data engineering as a foundational discipline.
We explore why data modeling—a once-vital skill—has fallen by the wayside and why it must be revived to support today’s complex data ecosystems.
Joe offers insights into the nuances of real-time systems, the significance of data contracts, and the role of governance in creating accountability and fostering collaboration. 

We also highlight two major book releases: Joe’s "Mixed Model Arts", a guide to modernizing data modeling practices, and our host Winfried Etzel’s book on federated Data Governance, which outlines practical approaches to governing data in fast-evolving decentralized organizations. Together, these works promise to provide actionable solutions to some of the most pressing challenges in data management today.  

Join us for a forward-thinking conversation that challenges conventional wisdom and equips you with insights to start rethinking how data is managed, modeled, and governed in your organization.

Some key takeaways:

Make Data Management tangible

• Data management is not clear enough to be understood, to have feedback loops, to ensure responsibility to understand what good looks like.
• Because Data Management is not always clear enough, there is a pressure to make it more tangible.
• That pressure is also applied to Data Governance, through new roles like Data Governance Engineers, DataGovOps, etc.
• These roles mash enforcing policies with designing policies.

Data Contracts

• Shift Left in Data needs to be understood more clearly, towards a closer understanding and collaboration with source systems.
• Data Contracts are necessary, but it’s no different from interface files in software. It’s about understanding behavior and expectations.
• Data Contracts are not only about controlling, but also about making issues visible.

Data Governance

• Think of Data Governance as political parties. Some might be liberal, some more conservative.
• We need to make Data Governance lean, integrated and collaborative, while at the same time ensuring oversight and accountability.
• People need a reason to care about governance rules and held accountable.
• If not Data Governance «(...) ends up being that committee of waste.»
• The current way Data Governance is done doesn’t work. It needs a new look.
• Enforcing rules, that people don’t se ant connection to or ownership within are deemed to fail.
• We need to view ownership from two perspectives - a legal and a business perspective. They are different.

Data Modeling

• Business processes, domains and standards are some of the building blocks for data.
• Data Modeling should be an intentional act, not something you do on the side.
• The literature on Data Modeling is old, we are stuck in a table-centric view of the world.

Chapters

• 0:20 History of Data Science
• 9:11 Data Engineering Discussed
• 22:55 Data Contracts
• 29:56 Data Governance
• 34:29 The Problem of Ownership
• 45:59 Data Modeling and Data Knowledge
• 52:58 Deeper understanding and learning