Why the Applied AI Layer Is Harder Than It Looks

Summary

The early critique of "applied AI" was that it would be a thin layer on top of foundation models — a temporary wrapper until models got good enough. What we're seeing in production tells a different story: driving agentic workflows in real enterprises is far more complex than surfacing tokens in a chat window. That complexity is where durable value gets built. And crucially, it does not shrink as models improve — it expands.

The shift

Across coding, legal, healthcare, customer support, and financial services, the pattern is consistent. Intelligence alone doesn't close the gap between a capable model and a workflow that actually runs in production. Intelligence must be bridged to real-world feedback loops — connecting to enterprise systems, getting the right data to the model, enabling humans to make decisions at different process steps through tuned UX, building workflows that improve the underlying data and models over time, and navigating regulatory and compliance constraints.

The amount of value that lives between the AI model and the end-user workflow is far larger than most assumed. Here is what that gap actually contains:

Workflow representation is not one-size-fits-all. Getting agents to work well — and alongside people — in mission-critical workflows requires representing agent interaction differently depending on the business process. Sometimes it is a chat experience. Other times it is a background agent running in a deterministic workflow. And dozens of other variants. This is a mix of needing a harness tuned to specific domains of work and making it show up in the right product experience.

Data context is vertical by nature. Different workflows connect into entirely different enterprise systems and need access to very different data. Working with that data — whether in life sciences, financial services, legal, or manufacturing — requires contextual approaches, deep understanding of the data schema, and the right user experience for data interaction. A single data integration pattern does not exist.

Domain-specific change management remains critical. The way you introduce technology at a bank is fundamentally different from a law firm. Having the right talent with a singular mission — people who understand both the technology and the industry — is essential for something as complex as process reengineering.

Model flexibility is a structural advantage, not a nice-to-have. The ability to work with a variety of models means you can tune workflows to different cost and performance levels. And you can eventually post-train models for specific tasks to tailor outcomes and eke out gains that are not coming from frontier models alone. This creates compounding returns for teams that invest in their eval and fine-tuning infrastructure early.

Evals have a crazy long tail. AI is not useful if it cannot be evaluated, and domain-specific evals that let you dramatically improve the performance of a harness for specific workflows span a virtually unbounded surface — there are too many distinct tasks in the economy for any single system to be tuned for all of them. The teams that build eval infrastructure per workflow are not solving a temporary problem; they are building into a permanently fragmented evaluation landscape.

Pricing models are part of the applied layer. Different verticals and domains require pricing models that reflect relevant abstractions on top of raw tokens. The ability to price in ways that match an industry's consumption model — per document, per case, per patient encounter, per reconciliation — matters as much as the underlying model capability.

Each of these six dimensions alone is a multi-month, multi-team effort to get right for a single workflow. Together they represent the full surface area of the applied AI layer — and they are why the gap between model capability and production value is so large.

The applied layer is not a temporary bridge — it widens as models improve. This is a critical and counterintuitive dynamic. The better models get, the more ambitious the workflows you can automate, and ambitious workflows demand more integration, more context engineering, more human-in-the-loop design, and more compliance handling. The applied layer does not shrink toward zero as frontier capability advances. It expands to fill the ambition that new capability unlocks. Client onboarding in a bank and contract review in a legal team will always require entirely different implementations — better models just raise the ceiling on what those implementations can achieve.

The opportunity is not just for the labs. Some of the applied layer will come from foundation model providers, but much of it will necessarily come from independent companies that can go deep in each industry. A single stack cannot absorb the variance between life sciences compliance, financial services regulation, legal workflow norms, and manufacturing operational constraints. Vertical specialization is not a stopgap — it is the structural reality of real-world AI deployment.

Why it matters

Leaders evaluating AI adoption should expect implementation depth, not a quick API integration. Budget and timeline for agent rollouts need room for workflow design, eval infrastructure, and organizational change — not just software licenses.

The counterargument — that model intelligence alone will eventually erase the applied layer — may hold in the limit. But the evidence points the other direction. Every major capability jump raises the ceiling on what workflows can be automated, and every new workflow brings its own integration surface area. The applied layer is not decaying; it is compounding.

The teams investing in workflow depth, eval discipline, and domain expertise now are building capability that compounds regardless of how fast models improve. And the independent companies that go deep in a vertical are not building a temporary moat — they are building the only kind that lasts when the frontier keeps moving.

What to do

• Scope agent projects around specific workflows, not generic "AI transformation"
• Invest in context capture and human-in-the-loop UX for the workflows that matter most
• Build or buy eval infrastructure per workflow — there is no universal agent eval suite yet, and the eval landscape is permanently fragmented
• Plan for change management as a first-class workstream, not an afterthought
• Stay model-neutral where possible; route by task economics, not vendor loyalty
• Bet against the thin-layer thesis: invest in integration depth, because it compounds with model improvement rather than being erased by it
• Design pricing models that match your vertical's consumption patterns — tokens alone are not a sufficient abstraction for most enterprise buyers