Teaching the Lakehouse to Speak Business: A Field Guide to Databricks Genie Ontology

The Context Gap in Enterprise AI

Large language models can write clean SQL syntax, but they have zero institutional memory. A model knows how to structure an aggregation or write a window function, but it does not know that your company defines gross margin differently for channel partners, or that accounts marked TRIAL_EXPIRED must be excluded from active customer counts.

When enterprise teams run into these errors, the standard response is to add dozens of rules into an agent’s prompt instructions. This approach quickly creates fragile prompts detached from the data pipelines underneath.

Reliable text-to-SQL requires an architectural solution. Databricks addressed this by expanding Unity Catalog beyond passive metadata storage for tables and permissions into an active context plane. At the center of this architecture is Genie Ontology: a structured knowledge layer designed to give AI tools the verified business background needed to return accurate, consistent answers.

The Two Tracks: Explicit Modeling vs. Inferred Behavior

Genie dynamically builds context on demand by pairing explicit rules set by data teams with inferred behavioral patterns captured across the workspace.

                          User Question
                               │
            ┌──────────────────┴──────────────────┐
            ▼                                     ▼
     Explicit Track                        Inferred Track
 (Human-Governed Truth)               (System-Learned Patterns)
  • Domains and Subdomains
  • Unity Catalog Pages                • Query History & Logs
  • Primary / Foreign Keys             • Dashboard Queries
  • Metric Views                       • Common Join Habits
            │                                     │
            └──────────────────┬──────────────────┘
                               ▼
                            OntoRank
            (Weights by authority, certification,
                  recency, and permissions)
                               ▼
                     Executable Spark SQL

The Explicit Track (Human-Governed Truth)

This track forms the authoritative backbone of your data assets:

Domains and Subdomains: Establish the top-level discovery perimeter. By grouping schemas and tables into logical business areas (like Sales Operations or Supply Chain), you prevent the agent from wandering across unrelated enterprise data.
Unity Catalog Pages: Attached directly to schemas and tables, Pages provide qualitative business documentation for rules that cannot be reduced to a formula, such as fiscal calendar structures or enterprise account criteria.
Metric Views: Defined in YAML or SQL, Metric Views centralize the computational ground truth for complex measures, dimensional grains, and ratios.
Informational Primary and Foreign Keys: Map the structural relationships between tables, showing the AI agent verified join paths and eliminating multi-table hallucinations.

The Inferred Track (Activity-Driven Context)

To prevent data teams from having to model every table by hand, the platform observes real-world workspace activity. It extracts common join habits, filters, and aggregations from scheduled jobs, production pipelines, dashboard queries, and past Genie conversations.

OntoRank: Deciding the Winner

When explicit definitions and inferred patterns conflict, OntoRank acts as the arbiter. It scores each context snippet using four criteria:

Source Authority: Explicit Metric Views and Pages naturally outrank ad-hoc query logs.
Certification Status: Assets tagged with Certified receive an algorithmic boost.
Usage Frequency and Recency: Fresh, frequently executed queries outweigh older patterns.
User Permissions: Context is trimmed at retrieval time based on the querying user’s access rights in Unity Catalog.

Jumpstarting the Cold Start: Ingesting Power BI Semantic Models

Building an enterprise ontology from a blank slate is an operational bottleneck. Most organizations, however, have already spent years refining their business calculations downstream inside Power BI tabular models.

Databricks addresses this cold start through direct metadata ingestion with Power BI. Unity Catalog connects to published workspaces, extracts measures, calculated columns, and table relationships, and converts them into live ontology snippets.

This integration acts as an immediate accelerator. Rather than spending months manually defining baseline metrics, teams can ingest a mature semantic layer on day one. As the system matures, engineers can progressively refactor those harvested measures into native Unity Catalog Metric Views upstream.

Governing the Genie Ontology Graph

An ontology is living software. It requires ongoing maintenance, clear boundaries, and predictable controls to prevent it from decaying as business priorities shift.

Algorithmic Tags: Steering OntoRank

In Databricks, metadata tags function as direct steering levers for query generation rather than passive labels:

Certified: Acts as a major relevance booster in OntoRank, ensuring official production tables, Metric Views, and Pages consistently outrank unvetted or inferred query patterns.
Deprecated: Actively suppresses legacy tables, staging schemas, and obsolete calculations from the agent’s reasoning path, preventing outdated business logic from resurfacing.

A Three-Step Triage Workflow

When an agent returns an incorrect answer, avoid patching the agent’s text instructions. Instead, resolve the error at its architectural source:

                          User Reports Incorrect Result
                                       │
            ┌──────────────────────────┼────────────────────┐
            ▼                          ▼                          ▼
     Wrong Table/Column            Wrong Math              Wrong Business Rule
             │                         │                          │
             ▼                         ▼                          ▼
   Update Unity Catalog       Update Metric View          Update or Author a
   Column Descriptions        (Formula / Grain)           Unity Catalog Page
   or PK/FK Constraints

Wrong table or join: Clarify column metadata, adjust domain boundaries, or declare missing primary and foreign keys in Unity Catalog.
Wrong calculation: Update the formula or aggregation logic in the central Metric View.
Wrong business rule: Update or create a Unity Catalog Page to clarify the policy in plain language.

The Real Payoff: Genie Ontology as a Shared Context Plane

Moving your organization’s business logic into Unity Catalog turns the lakehouse into a shared context plane. Rather than repeatedly patching dashboard filters or answering one-off calculation questions in Slack, data engineers become curators of organizational knowledge that powers internal analytics, autonomous agents, and downstream applications from a single source of truth.

Building a reliable ontology requires the right balance of data modeling, governance, and platform architecture. If your team is looking to design, audit, or accelerate your Databricks semantic layer, contact our team at Entrada to schedule an architectural working session.