The Signal/Data & Infrastructure

    Data & Infrastructure

    TheDataLayerIsWhyEnterpriseAgentsFailinProduction

    22 June 2026 · 5 min read · By En Interactive

    Data & Infrastructure

    Databricks opened its Data + AI Summit on June 16 with a deceptively simple claim: the agent loop — the LLM reasoning core that occupies most enterprise AI investment — represents 1% of the work required to run agents in production. The other 99% is surrounding infrastructure. Three days of announcements followed, designed to make the case that Databricks has built the missing 99%.

    What Actually Happened

    At DAIS 2026, running June 15–18 in San Francisco, Databricks announced a cluster of infrastructure changes centered on agent readiness. Lakehouse RT, powered by a new compute engine called Reyden, delivers sub-100ms query latency directly on governed Delta Lake and Iceberg tables — eliminating the separate real-time serving tier that enterprise teams typically maintain alongside the lakehouse. According to Databricks' release blog, the product performs up to 16x faster than separate real-time serving stacks, with no additional change-data pipelines required.

    LTAP (Lakehouse Transactional Analytics Platform) pairs Lakebase — a serverless Postgres database — with the Reyden-powered analytics layer, keeping both operational and analytical workloads on a single copy of data in open formats. The Agent Bricks platform expanded into a full developer platform, with Unity AI Gateway adding governance, per-user and per-agent cost controls, and automatic model routing. Genie Ontology — a context layer that continuously learns organizational semantics — enables agents to understand what terms like "margin" or "churned customer" mean in a specific business without re-establishing context on every call.

    According to Moor Insights & Strategy analyst Mike Leone, writing on June 18, Lakehouse RT removes an entire infrastructure tier: "Real-time serving has usually meant a separate, specialized store next to the lake, with replication keeping the two in sync... Reyden runs inside Unity Catalog with the same governance as everything else, no separate copies and no change-data pipelines."

    The Question Nobody Is Asking

    Coverage of DAIS 2026 focused on product names. The structural argument underneath deserves more attention.

    Enterprise agents fail in production at a predictable point, and it isn't the reasoning step. An agent that queries a customer record, writes an observation to memory, immediately queries that updated record, and triggers a downstream workflow is doing something fundamentally different from a dashboard. It issues sub-second reads and writes, requires millisecond latency on arbitrary columns, produces structured output that must immediately become queryable input, and does this thousands of times per hour across parallel sessions. Most enterprise data stacks were built for human access patterns — batch ETL pipelines that land data overnight, a separate OLTP store for transactional writes, and a real-time serving layer maintained via CDC pipelines. Every one of those architecture seams is a failure point for agents at scale.

    Databricks is arguing — with LTAP, Lakehouse RT, and a unified governance layer — that the answer is a single data tier serving operational, analytical, and agent workloads simultaneously on the same storage. That premise is grounded: AstraZeneca, 7-Eleven, and Block have shipped production agents on Agent Bricks, and the infrastructure constraints those deployments exposed are precisely what DAIS 2026 addressed.

    What hasn't entered the coverage is the context layer risk. Genie Ontology addresses a real problem: agents that understand organizational semantics require far less context injection per call, reducing latency and cost. But a layer that learns from how people use the data concentrates errors — if the most common usage of a business term is wrong, the ontology systematizes the mistake. The correction mechanism for that scenario is the primary evaluation question Genie Ontology doesn't yet publicly answer. For regulated industries where definitions of "eligible customer" or "reportable event" carry compliance weight, this matters before deployment, not after.

    The Enterprise Lens

    Two architecture decisions are worth revisiting for teams with agents in production or active development.

    First, assess whether your real-time serving setup can be simplified. If you are running a separate OLTP store or low-latency tier alongside a lakehouse — and maintaining CDC pipelines to keep them synchronized — Lakehouse RT directly replaces that topology. The relevant test is whether Reyden's performance on your specific data shapes and query patterns matches published benchmarks.

    Second, evaluate LTAP against any workload where agents read their own outputs as inputs within the same session. This is the failure mode LTAP directly addresses: an agent that writes a decision and immediately queries it through an analytical layer that hasn't received the write will produce incorrect downstream reasoning. If your agent architecture involves that pattern — and most production agentic systems do — the unification LTAP provides has direct relevance.

    What to Watch

    • Whether LTAP's single-copy architecture holds under concurrent read/write production loads — the architectural premise is sound; independent validation at production concurrency is what establishes it as a real enterprise choice
    • How quickly Unity AI Gateway's per-user and per-agent cost controls become standard expectations in enterprise AI infrastructure RFPs, and whether competing platforms adopt equivalent governance within the next two quarters
    • Whether Genie Ontology's correction mechanisms prove sufficient for regulated industries, where an agent that systematizes an incorrect business definition creates audit exposure rather than quality variance

    Sources

    #Databricks#Data Infrastructure#Agentic Systems#Lakehouse RT#Unity AI Gateway
    Back to The Signal