Expedia Group’s first chief AI and data officer, Xavi Amatriain, delivered a clear message to enterprise leaders at the VB Transform 2026 conference: the era of the static product requirements document (PRD) is over. In its place, he argues, engineering teams should be writing evaluations — evals — as the primary specification for what a product should do. “The new PRD are the evals,” Amatriain told the audience in Menlo Park last week. “You encode what you want the product to do through your evals, which might include red teaming evals and all kinds of other things, which already have a bunch of security requirements. So, you already embed that into the PRD and the product design document before you even start coding.” He pushed the argument further, stating that in a future dominated by AI-assisted and AI-generated code, “all your thinking is gonna go into the evals.”
Amatriain, who previously served as VP of AI and Compute Enablement at Google, where his work underpinned platforms for Gemini and Google Search, was appointed to his role at Expedia in December 2025. He has also mentored talent that later founded companies like Perplexity and Scale AI. His perspective arrives at a critical moment for the industry. VentureBeat’s own Pulse research on the evaluation gap found that 66 percent of the 157 enterprises surveyed already permit some production deployment without human review or are building toward that capability within the next 12 months. Yet only 5 percent fully trust the automated evaluations that would make that decision. Half of the respondents have shipped an agent that passed internal evals but then failed with a real customer.
The Case for Minimizing Guardrails
Amatriain challenged a common enterprise instinct to layer rigid rules onto AI systems. “The more guardrails and artificial business rules and sort of rules that you put into the system, the worse off,” he said. “Not only because they’re brittle, but also because they actually mess up with the feedback loop. You are actually biasing the user and the feedback you get from the user, and then you’re learning that in the wrong way.” He described guardrails as “a necessary evil” and said the goal should be to minimize their impact over time. This position was not universally accepted at the event, where other speakers argued that the highest-risk actions still demand very firm guardrails.
Expedia’s approach to governing AI relies on three distinct layers rather than on brittle rules. Principles come first and are communicated broadly across the organization. “I like to encode at a very high level how I expect decisions to be made, because in a large organization you’re gonna have a lot of distributed decision making,” Amatriain explained. Processes and tools that enforce those principles follow, giving them teeth. Automation sits on top of both layers. In practice, this plays out through what Expedia calls agent release toll gates — checkpoints calibrated to risk. “Governance needs to correlate to the risk,” Amatriain said. The toll gates tie evaluation rounds, red teaming, and security review to each agent’s risk level, shifting from recommended to required as the stakes climb.
Specialized Agents and a Compositional Architecture
Expedia’s technical strategy is built on a composition of specialized agents rather than a pursuit of a single, monolithic general intelligence. “Even when I was at Google, I was like, I don’t believe in AGI as sort of like a singleton and a unified sort of like single model,” Amatriain said. “I think it’s much better to think of it as composition, sort of like having specialized agents that are very good at some task and then composing the system out of those specialized agents.” The architecture starts at the component level: tools compose into skills, skills assemble into sub-agents, and sub-agents get orchestrated into the full agentic system. This requires unified design principles governing tone, user addressing, context passing, and memory. “All of that needs to be thoroughly designed,” he said, framing the challenge as a systemic design problem rather than a model selection exercise.
An important benefit of this approach is security. Scoping each agent narrowly allows teams to evaluate and lock down individual components in isolation before composing them into a larger system.
Why the User Must Keep the Final Click
Travel is a domain where real-time pricing, shifting flight availability, and conflicting supplier claims create a uniquely challenging environment for AI agents. Amatriain described a system that blends retrieval-augmented generation with direct API tool calls, choosing the approach based on latency. A question about the average cost of a four-star hotel in Chicago in July can be answered from a cache in seconds. A query about a pet-friendly four-star near Lake Michigan with a pool might justify a 30-second reasoning window to cross-reference reviews against supplier data. “The supplier might be saying, yeah, we have a great swimming pool, but then we also have the reviews from the travelers and we actually see there’s two reviews that say the swimming pool was not great or was not open after 6 p.m.,” Amatriain explained.
Critically, the agent is designed to stop short of taking the final action. “We don’t want the agent to book the hotel or to buy you a plane ticket for you,” Amatriain said. “That’s something that the user has to have the agency. And the agent can recommend, can suggest, can discuss with you, but you’re gonna have to hit that click. And that’s non-negotiable.” He framed this design principle as both a user experience and a security decision, noting that establishing it early removes the need for elaborate guardrails added after the fact.
The Growing Threat from AI-to-AI Attacks
When asked about security lessons from production, Amatriain emphasized that security must be shifted as left as possible and embedded in design. “Usually when you need a guardrail is because you’ve not thought about it early on,” he said. He described a feedback loop where monitoring signals from the operating AI system flow back into the eval suite, enabling a nearly automated cycle of detection, reporting, and fixing. “Having that whole feedback loop from real signals, from your operating AI system, all the way into being reported and fixed as quickly as possible is going to become essential,” he said.
The margin for error is thin. VentureBeat’s separate Pulse survey on agent security, drawn from 107 enterprises, found that 54 percent have already had an agent security incident or near-miss. Incident rates reach 63 percent among enterprises with more than 1,000 employees. Amatriain warned that the threat landscape is evolving. “You’re gonna get threats coming not only from humans but also from other external agentic systems that are really powerful, and they’re gonna be poking at everything you’re doing. And as soon as you detect something, it’s not only about the detection, but the time to fix becomes essential here.”
What This Means for Engineering Teams Now
Amatriain’s framework offers a practical starting point for any team building production AI agents. The first step is to reframe the development workflow: invest in writing and maintaining a comprehensive eval suite as the primary specification, not as an afterthought. The second is to establish a governance model that calibrates oversight to risk, using principles, processes, and automation in sequence rather than relying on brittle guardrails. The third is to design agentic systems as compositions of narrowly scoped, individually evaluable components. Teams that adopt this approach will be better positioned to close the evaluation gap that most enterprises still face, reducing the risk of shipping agents that pass internal tests only to fail in the hands of real users. The time to begin is now, before the next generation of AI-powered threats — and competitors — force the issue.
