Working backwards from the end goal is a core principle of software development, and we've found it to be highly effective in modelling data products. In this article we'll explore a step-by-step, methodical approach to identifying data products that avoids overdesign while providing just enough clarity for teams to begin implementation.
ReadDecentralized data management requires automation to scale governance effectively. Fitness functions are a powerful automated governance technique we've applied to data products within the context of a Data Mesh. Since data products serve as the foundational building blocks of a data mesh, ensuring robust governance around them significantly increases the chances of a successful data mesh transformation.
ReadIn the technology stream of our discovery process, data engineers engage with the domain that's being onboarded to understand their existing platform capabilities and the scope of any data products they already have in place. This helps identify the Data Mesh delta that will need to be bridged with new technology and architecture.
ReadIn this article, we dive into the first of those streams, looking at the operating model changes required to support Data Mesh, and the discovery process that helps us identify and define them.
Read