Blog by Kiran Prakash

Function calling using LLMs

Tue, 06 May 2025 00:00:00 GMT

While LLMs excel at generating cogent text based on their training data, they may also need to interact with external systems. Function calling allows them to construct such calls. The LLM does not execute these calls directly, instead it creates a data structure that describes the call, passing that to a separate program for execution and further processing. The LLM’s prompt includes details about possible function calls and when they should be used.

…read the full article here.

Designing Data Products

Tue, 10 Dec 2024 00:00:00 GMT

Working backwards from the end goal is a core principle of software development, and we’ve found it to be highly effective in modelling data products. In this article we’ll explore a step-by-step, methodical approach to identifying data products that avoids overdesign while providing just enough clarity for teams to begin implementation. Starting with a use case, we work backward to define data products and establish their boundaries. By overlaying additional use cases, we generalise the data products to prevent overfitting. Finally, we assign domain ownership and define service level objectives to guide the architecture and implementation.

…read the full article here.

Governing Data Products Using Fitness Functions

Thu, 05 Sep 2024 00:00:00 GMT

Decentralized data management requires automation to scale governance effectively. Fitness functions are a powerful automated governance technique we’ve applied to data products within the context of a Data Mesh. Since data products serve as the foundational building blocks (architectural quanta) of a data mesh, ensuring robust governance around them significantly increases the chances of a successful data mesh transformation. In this article, we’ll explore how to implement this technique. We’ll start by designing simple tests to assess key architectural characteristics of a data product, and then we’ll explore how to automate their execution by leveraging metadata about the data products.

…read the full article here.

Data Mesh in practice - Technology and the architecture (Part IV)

Thu, 19 May 2022 03:00:00 GMT

In the technology stream of our discovery process, data engineers engage with the domain that’s being onboarded to understand their existing platform capabilities and the scope of any data products they already have in place. This helps identify the Data Mesh delta that will need to be bridged with new technology and architecture, and what new data products should look like from a technical perspective.

…read the full article here.

Data Mesh in practice - Product thinking and development (Part III)

Thu, 19 May 2022 02:00:00 GMT

When teams want to start treating data as a product, we recommend working backwards from organizational goals to identifying high-value analytical use cases, and ultimately, which data products are needed to bring the use cases to life.

…read the full article here.

Data Mesh in practice - Organizational operating model (Part II)

Thu, 19 May 2022 01:00:00 GMT

In this article, we dive into the first of those streams, looking at the operating model changes required to support Data Mesh, and the discovery process that helps us identify and define them.

…read the full article here.

Data Mesh in practice - Getting off to the right start (Part I)

Thu, 19 May 2022 00:00:00 GMT

In this series of articles, we’ll share practical learnings from our recent Data Mesh implementation engagement at Roche. As the world’s largest global provider of healthcare, Roche generates, manages, and processes vast quantities of data across a wide range of deeply specialized domains. We lay out what it really takes to successfully apply the principles of Data Mesh in a large corporate environment.

…read the full article here.

The Curse of the data lake monster

Wed, 20 Feb 2019 23:46:37 GMT

Artificial intelligence and machine learning are currently all the rage. Every organisation is trying to jump on this bandwagon and cash in on their data reserves. Organisations are investing a lot in building fancy data lakes but often without considering the use cases. This approach could result in a lot of wasted effort and is also the primary reason why a lot of data lake initiatives fail. In this essay, Lucy and I explain a better approach.

…read the full essay here.

The Curse of the data lake monster

Wed, 20 Feb 2019 23:46:37 GMT

…read the full essay here.