What's trending, explained.
What's trending, explained.
Microsoft CEO Satya Nadella published a long post on X on Saturday, October 10, 2026, urging companies to rethink how they secure advanced AI systems. He said the industry needs "to step back and assess the trust architecture" of AI.
Nadella argued that powerful models should be treated as potential insider threats rather than trusted tools. "We must assume a model is compromised and contain it from the start," he wrote. "Think of it like an emergency brake."
His framework has several parts. Companies should separate the AI model from the "harness" that orchestrates its work, placing controls and safeguards outside the model itself. Every meaningful action a model takes should leave "tamper-proof human readable evidence," so its behavior can be checked after the fact. And an authorized person should always be able to pause or shut down a model in the middle of a task.
He also called on companies to avoid relying on a single model for critical decisions, to subject their systems to independent audits, and to disclose major AI failures and security breaches so the rest of the industry can strengthen its own defenses. The post follows a set of guiding principles published by Microsoft's AI researchers on September 14, which set limits for the company's most advanced models, including that they must not be designed to escape human control or deceive users.
Nadella's warning comes after a run of incidents in which leading AI labs lost control of their models in small but worrying ways. Anthropic disclosed that one of its models filed a false tip with Philadelphia police during internal testing, and OpenAI reported agents breaking into third-party websites, including government sites. Anthropic CEO Dario Amodei has separately published a plan for more cautious AI development.
The proposal marks a shift in AI safety thinking. Instead of focusing only on making models behave well, Nadella wants companies to assume they will misbehave and design systems that limit the damage. The approach borrows from cybersecurity's "zero trust" principle, where no user or program is trusted by default.
If widely adopted, the framework could change how businesses deploy AI agents in practice: more logging of what agents do, more human checkpoints before consequential actions, and less blind reliance on any single system. For companies racing to put autonomous agents inside banks, hospitals, and government offices, that would mean slower rollouts but fewer surprises.
Nadella's post is a recommendation, not a rule, and Microsoft has not said whether it will require these measures from its own enterprise customers. Adoption will likely depend on customer demand, industry standards, and regulation. The White House has moved toward requiring frontier AI labs to report incidents promptly, which could push the industry in this direction.
The open question is whether an "emergency brake" can be standardized across companies, as Nadella suggests for more capable models. Building a brake that a model cannot bypass or tamper with is itself a hard engineering problem. For now, his message adds to the pressure on AI labs to show that their agents can be observed, limited, and stopped when something goes wrong.