Entrada

Every conversation with a CIO this year ends the same way: “Do we still need Power BI if we have Databricks Genie?” The honest answer is more interesting than yes or no. Here is what I tell clients before they rip out a working BI stack.

Most Databricks ML pipelines do not fail because the math is wrong. They fail because performance decisions made early quietly compound until cost, latency, and trust all start slipping at once.

After years of architecting ML platforms on Databricks, one pattern keeps repeating: the difference between a model that survives in production and one that quietly fails usually comes down to how features are managed. Here’s what we’ve learned the hard way.

Every serious governance project eventually reaches the same uncomfortable moment: the platform has the metadata, but the organization still does not have a product. There is a catalog. There are tags. There are comments, owners, lineage events, audit rows, dashboards, policies, and a dozen local rituals around who is allowed to change what. Yet when a steward asks, “Can I safely change this field?”, the answer still arrives as a meeting, a spreadsheet, and a prayer.

Inside the Customer Billing Accelerator from Entrada and Databricks, an agentic AI stack that detects anomalies, answers finance questions in plain English, and writes back to source systems, all governed through Unity Catalog.

A field report on what changed between DataPact 2.9 and 3.0, why we put a managed Genie space at the centre of the release, and the engineering it takes to make a conversational data quality surface trustworthy enough to call a product.

This post walks through the architecture Entrada built around that observation, the Serverless Cost Control Accelerator, and, more importantly, the design principles behind it. Regardless os whether we’re a platform engineer, SRE, or FinOps lead trying to decide where to invest, the principles matter more than the product.

Most ML models do not break in development. They break quietly in production, when data changes, performance drifts, and no one notices until business trust is already slipping. That is why monitoring is not an afterthought. It is one of the foundations of enterprise AI.

Data engineering teams spend enormous energy building reliable pipelines – clean medallion layers, solid transformation logic, well-tuned Spark jobs. Then something breaks in production that worked perfectly in development. A library version changed. An environment variable was missing. A Spark executor launched with a subtly different runtime than the one the job was built against.

When I work with high performance teams, whether in motorsport or enterprise, I see the same pattern: data is not the advantage. The advantage is the ability to turn data into decisions that are fast, trustworthy, and repeatable.