Of all the debates raging about the potential downsides of artificial intelligence, a singular concern has begun to dominate conversations among Silicon Valley’s most seasoned technologists. The fear is that the giant labs behind proprietary AI models, including OpenAI and Anthropic, are operating as Trojan horses. As startups and enterprises feed these models their most sensitive business information to improve outputs, the labs gain an ever-expanding view into the proprietary knowledge that defines a company’s competitive edge. The ultimate risk is that model makers could use that knowledge for themselves, effectively becoming competitors to their own customers. Now, in a surprising intervention, Microsoft CEO Satya Nadella has formally joined this warning chorus, arguing that AI users are “paying twice.”
Nadella’s Core Argument: You Pay with Money and Data
In a blog post published on Sunday, Nadella lays out a stark thesis. He argues that enterprises knowingly pay for AI token usage, but they also, often obliviously, hand over something far more valuable in the process: their proprietary business data. “You essentially pay for intelligence twice, once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful,” he writes. The better a model performs, the more of that knowledge a company has to feed it, creating a deepening dependency that Nadella views as fundamentally asymmetrical and dangerous.
This vulnerability extends beyond simple input data. Nadella warns that models learn from “exhaust,” which includes the prompts people write, the tools agents use, and, most critically, the corrections users make when a model is wrong. “Every correction is distilled into institutional know-how,” he explains. This is the kind of knowledge a competitor could never buy, yet enterprises are handing it over willingly in the course of improving their AI workflows.
The Distillation Debate: A Hypocritical Asymmetry
Nadella points to a critical hypocrisy in the current AI ecosystem. Model makers argue for fair-use rights to train on all publicly available internet data. Yet, Nadella notes, they simultaneously impose restrictive terms on “distillation”—the practice of using a model’s own outputs to study its behavior and train a new, often cheaper, model. In February, Anthropic accused Chinese open-source labs of sending millions of prompts to its Claude model specifically to improve rival systems, urging the U.S. government to tighten export controls.
Nadella’s response is pointed: model makers cannot have it both ways. “While the great innovation that comes from model providers having fair use rights to train models on public data is needed, I find it ironic that the status quo is to then turn around and impose restrictive terms on distillation,” he writes. The CEO is particularly concerned when model providers “reserve the right to learn from customer usage and interaction data,” a common clause in many proprietary AI service agreements.
The Enterprise Response: A Shift Toward On-Premise Open Source
Nadella’s proposed solution is predictable for the CEO of a giant cloud provider. He urges companies to “retain ownership” of all data, including prompts and feedback, and to build their own “proprietary learning environments” in the cloud. He also recommends creating what he calls “orchestration layers” to allow easy switching between AI models from different providers, preventing vendor lock-in. While Nadella never explicitly uses the term “open source,” it is the obvious subtext of his argument.
Industry evidence suggests this shift is already underway. Idit Levine, founder and CEO of Solo.io, a company that provides networking and security software for managing AI systems, reports seeing this exact transition play out with her clients. After experimenting with proprietary models, companies begin to ask, “Can I take an open source model and run it on-prem? It will do almost 90% of what the big one’s doing. It will cost way less,” she explains. “They understand that, and they can control it.” Solo.io, whose technology powers the Linux Foundation’s Agentgateway project, counts T-Mobile, ADP, and SAP as customers.
Vercel and OpenRouter, both companies that provide AI model-switching tools, are also seeing a surge in traffic to open-source models. Open models accounted for 29% of all traffic routed through Vercel’s gateway last month, a number that is likely to grow as enterprises seek greater control over their data and AI infrastructure.
What This Means for AI Adoption and Data Strategy
Nadella’s warning represents a significant shift in the public posture of a company that has invested heavily in both OpenAI and Anthropic. His core message—that by consuming intelligence, you are creating intelligence, and what you create should belong to you—is fundamentally a call for data sovereignty. For organizations evaluating their AI strategy, the practical implication is clear. Any enterprise using a proprietary model should carefully review the provider’s terms regarding the use of prompt data, corrections, and interaction logs. Tools like AI gateways and orchestration layers are no longer optional technical features; they are a necessity for maintaining control over proprietary knowledge.
Enterprises should now treat AI model selection as a data governance decision first and a cost or performance decision second. The organization that owns and controls the data used to fine-tune and correct its models will be the organization that retains its long-term competitive advantage.
