Satya Nadella on AI Alignment and Zero Trust

In 2018, Satya Nadella believed what most of the field believed: that if you scaled a model up, capability and safety would arrive together, like two passengers on the same train. He says so himself, naming the believers as "Dario and Ilya and Sam and Greg and Elon," back when they were, in his words, "one happy family" (00:00). Dario Amodei now runs Anthropic. Ilya Sutskever co-founded OpenAI and later Safe Superintelligence. Sam Altman runs OpenAI. Elon Musk co-founded OpenAI before leaving to build his own lab. They scattered, and so, it turns out, did the assumption that held them together. Nadella, the CEO of Microsoft, is now building an entire security architecture around the fact that it was wrong.

"Scaling laws are working," he tells podcast host Harry Stebbings early in the conversation. "Except one of the assumptions I had was that you'll have alignment just fall out of scaling, and it doesn't seem to be" (03:33). Alignment, in this context, means getting an AI system to actually do what its designers intend, rather than something that merely looks competent. Nadella adds that researchers still lack a mechanistic understanding of how these models work on the inside (03:54). That is an unusually blunt admission from the head of a company that has bet its future on this technology. His response is not to wait for the science to catch up. It is to build a cage around the uncertainty.

A Cage, Not a Cure

That cage is what Nadella calls a zero trust architecture, a security approach that assumes no system or user is automatically trustworthy and checks everything continuously. He separates this sharply from the unsolved problem of AI safety itself.

"Let's create an engineered zero trust environment that can contain any malfunctioning agent before it does any harm." — Satya Nadella [04:14]

He does not think reinforcement learning, a training method that rewards a model for good outputs after the fact, can get a model to true alignment on its own. Alignment, he argues, has to be built into the model's internal parameters (its weights) from the start, ideally during the earliest phase of training called pre-training.

"I don't think you can even RL your way to alignment. You have to truly get alignment to be something that truly gets embedded in the weights." — Satya Nadella [04:56]

Until that happens, Microsoft is shipping the containment system instead. Two tools, Agent 365 and Perception, give every AI agent inside a company a traceable identity and log its tool calls, its model calls, and its chain of thought, the step by step reasoning a model produces before answering (05:16). Nadella wants that reasoning written in plain English, not some private shorthand a model invents, and he wants the logs tamper proof so an agent cannot edit its own record (05:36). Policies get set on top, and the system watches for any agent that strays from them (06:22).

The catch is that the thing being monitored is not fully trustworthy either. Nadella concedes that a model's chain of thought can be "unfaithful," meaning the reasoning it shows is not necessarily the reasoning it actually used, and that this gets worse if a model's own maker decides to hide it (06:50). That tension, between wanting transparency and not fully being able to verify it, is quietly reshaping his view of the market. "Closed weight models are more powerful, but if they kind of don't recognize the need to be more transparent, especially as their trusted partners use it, then we may have a problem," he says (07:13). Closed weight models are proprietary systems whose internal parameters the public cannot see; open weight models publish those parameters for anyone to inspect or modify.

Asked to predict where token usage, meaning the volume of text processed by AI systems, will flow over the next 24 months, Nadella says most of it will become commodity usage, cheap and interchangeable, whether that commodity turns out to be open weight or merely inexpensive closed weight models (08:47, 09:48). Frontier models, the most capable and expensive systems, will persist mainly for problems at the edge of what is currently possible, such as scientific research. Tellingly, Nadella says he wants Microsoft's own MAI models to behave less like a model locked to a single company's network and more like ordinary software that customers can license and run. "I want to turn this into just a simple, straightforward software business as opposed to a network effects intelligence business," he says (08:21).

Who Owns the Intelligence

If models become commodities, the next question is where value actually accumulates, and Nadella's answer reaches back to two old economists. He invokes Austrian economist Friedrich Hayek's idea that useful knowledge in an economy is scattered across countless individual firms and people, not concentrated in one place (10:48). If AI lets one firm or model absorb everyone else's tacit knowledge, that scattering collapses, and so, he argues, does the price mechanism that makes markets work.

He then borrows from economist Kenneth Arrow's classic paradox of information: a seller who reveals a secret to prove its value has already given it away for free, patent or no patent (11:09). Nadella inverts it for the AI era. The party now at risk of leaking value, he says, is the buyer, the company using an AI model, because every interaction, every thumbs up or thumbs down, becomes "exhaust" that trains and improves the model provider's product rather than the buyer's own capability (11:36). He calls this the reverse information paradox.

"A pricing mechanism works because of that. So therefore, I think it'll be important for every firm to be able to accumulate knowledge, at increasing scales using AI versus leak." — Satya Nadella [11:09]

His proposed fix is for companies to build their own evaluation benchmarks, meaning internal tests that measure how well a model performs on that specific company's real tasks, and to treat those benchmarks as proprietary. "Every firm will have a private eval benchmark, which is their core IP, and every model comes in and tries to sort of hill climb on that eval, and that to me is how a firm works," he says (12:08). He expects the label "AI native," currently reserved for a handful of forward leaning companies, to feel as dated within five years as "cloud native" does today, pointing to Lloyd's, the 500 year old London insurance market, as an example of an old institution that eventually became cloud native without anyone remarking on it (13:22, 13:38).

That diffusion, Nadella argues, will create what he calls an implementation economy, a layer of work that blends classic business consulting with technical, agentic engineering, neither of which fully existed before (14:17). He points to Unilever, the consumer goods company, which built an agentic system to automate order to cash processing and trade claims, started on a frontier model, then switched much of the work to a cheaper Microsoft MAI model it could train on its own data (14:34). On the bigger structural question of whether one company will eventually control the entire stack from chips to applications, Nadella leans against it. "If anything, it's going the other way, which is every day you wake up and you say, wow, there's more competition in every layer of the stack," he says, predicting multiple winners at each layer rather than one vertically integrated victor (16:43).

Underneath all of this sits a wager on economic growth. Mustafa Suleyman, CEO of Microsoft AI, has warned that frontier training runs could reach $100 billion (17:22). Nadella frames that figure simply as a share of corporate research and development spending, noting that heavy R&D companies already spend 15 to 20 percent of revenue that way (17:48). The money only works, he says, if AI lifts world GDP growth off its long standing plateau of roughly 2 percent a year (18:53), largely by cutting costs in sectors like healthcare and by absorbing what he believes is a much larger backlog of unmet software demand than anyone previously recognized (19:45).

The conversation ends somewhere smaller and more personal. Nadella has walked on an inclined treadmill every morning for about 25 years, run interrupted only by his knees giving out (21:44). Asked what he has changed his mind about in the last year, he returns to where he started.

"I used to always think that alignment will just fall out. I no longer believe that." — Satya Nadella [22:31]

It is a rare thing to hear the head of a trillion dollar company admit that a core assumption behind his biggest bet has quietly failed, and then describe, in granular detail, the scaffolding he is building to compensate for it. His hope, stated plainly at the end, is that the upside outruns the uncertainty: that AI diffuses faster and further than any technology before it, including into the Global South, a term for lower income countries largely outside North America and Western Europe, where his own father worked as a development economist (23:56). Whether that hope survives contact with $100 billion training runs and reverse information paradoxes is, by Nadella's own account, still an open question.