Microsoft’s outage started with a fix from July. Resilience is maintenance, and it needs a budget.
Microsoft’s preliminary report traces last week’s Microsoft 365 outage to a defect shipped in July that its load tests missed, triggered by lost capacity and an automatic failover, and spread by the systems meant to contain it. Every safeguard was built for a smaller failure. For customers, the lesson is concentration: email, chat, documents and now AI assistants share one dependency, and the fallback is usually nobody’s job.
What happened
On 12 and 13 November, Microsoft 365 suffered a broad outage that disrupted Outlook, Exchange Online, Teams, SharePoint, OneDrive, Office apps and Copilot. Downdetector logged reports from 74 countries and territories, according to the AP. Microsoft said the cause was “not related to cyber or AI / agents”.
Microsoft published a preliminary post-incident report to customers on 15 November, and the reporting by TechCentral and The Next Web sets out the chain of events:
- July. Microsoft changed the servers in a network gateway that sits in front of many Microsoft 365 services, including Exchange. The change had a latent defect: under certain loads, internal Windows tasks blocked each other until the servers ran out of processing power and memory. It passed load testing, because the tests ran on weaker hardware with fewer processor cores, where the fault rarely shows.
- Three months of normal running. The change ran worldwide without incident.
- 11 and 12 November. An unrelated problem took nearly 14% of the gateway’s capacity offline. Early on 12 November, Microsoft’s systems automatically moved Teams traffic onto the gateway that carried Exchange Online. The lost capacity and extra load, during peak US traffic, triggered the defect.
- Spread. Traffic management moved users off failing servers onto healthy ones, which then failed the same way. Within two hours more than 80% of the gateway’s servers were unresponsive at the low point. Barriers meant to keep failures inside a region did not hold, for a reason the report does not yet explain.
- Slow recovery. Servers that came back were overwhelmed by queued traffic and failed again. Microsoft said its automated recovery was not designed for a failure this broad, and that normal rollback, flighting and feature flags could not quickly undo a months-old change. Engineers wrote a new update to restore the earlier code path and applied it group by group from the morning of 13 November. Email recovered last, because Exchange Online could not move off the failing gateway “for scale reasons”.
Microsoft has committed to monitoring how traffic is redirected, limits on how load is redistributed and added capacity, with a final report due on 20 November. The chair of the House Homeland Security Committee, Andrew Garbarino, said he was seeking more information from Microsoft and CISA.
My view
Microsoft has been unusually open about this, and the preliminary report is worth reading in full. My view is less about Microsoft than about what every firm can learn from it.
Every safeguard worked as designed. The design assumed a smaller failure.
Failover moved load onto the weak point. Traffic management spread the fault from server to server. Rollback assumed the bad change was recent. Each mechanism is sensible for the failures it was built for, and each made this one worse. Resilience that has only been tested against the common failure can amplify the rare one. The useful question for any critical system is not “do we have failover?” but “what does failover do when the target is already sick?”
The tests didn’t look like production
The defect slipped through because the load tests ran on hardware that didn’t match the newer, larger machines in production. That is the same failure I see in data and AI work: pipelines tested on clean samples, models evaluated on data that doesn’t look like next month’s, access rules checked against a test directory instead of the real one. A test environment that doesn’t represent production is a hygiene problem, and it produces exactly this outcome: a change that passes, ships, and waits.
Latent defects are debt nobody priced
A change that sits quietly for three months is technical debt with an unknown due date. The work that finds it before it fails, production-scale testing, recovery drills for broad failures and removing old paths, doesn’t ship a feature, so it competes badly for budget. That is true at Microsoft’s scale and it is more true inside most enterprises, where resilience testing is the line item cut first. Microsoft’s final report on 20 November will say what it is changing. Most firms will never write that report for their own systems, because nobody is asked to.
Your concentration risk is yours
For customers, the bigger lesson is how much now runs on one dependency. Email, chat, documents, identity and, increasingly, AI assistants and agents sit on the same provider. When it fails, the workflow and the people who would run it by hand both lose their tools at once. Banks already have a frame for this in third-party risk guidance, and in Europe DORA asks financial firms to plan for the failure of critical ICT providers. In practice the manual fallback for “email is down for a day” is rarely written, tested or owned.
Agents deepen the dependency
As firms move work onto agents, an outage stops more than communication. An agent that drafts replies, files cases or reconciles accounts through the same identity, mail and storage stops with it, and the queue it was clearing grows. Every agent workflow needs a defined degraded mode: the agent stops cleanly, the work queues safely, and people have a procedure and the access to follow it.
Why it matters
For the industry. Outages, breaches and data errors usually get separate post-mortems from separate teams. They share causes: changes tested against conditions that don’t match production, safeguards built for the common case, dependencies nobody mapped and maintenance nobody funded. Firms that treat resilience as maintenance with an owner and a budget will recover faster from the next one, whoever causes it.
How it connects to our work. The Reference Implementation asks every capability to define its degraded mode, usually the manual process it replaced, and its recovery time. Build It to Run treats build and run as one loop, so every incident sends a change back into the build. Choosing an Enterprise Platform for AI Agents covers portability and exit. Five steps I’d take first:
- Map what shares one provider. Email, identity, documents, chat and AI assistants. Know which critical processes stop together.
- Write and test the fallback. For each critical process, decide how it runs for a day without the provider, who decides to switch, and practise it.
- Test failover against a sick target. Ask what your failover and traffic rules do when the destination is already degraded.
- Make test environments look like production. Hardware, data volumes, real directories and real data shapes, for software, data pipelines and AI evaluations alike.
- Give every agent a degraded mode. Stop cleanly, queue safely, and hand the work back to people with the access to do it.
This is one of three Viewpoints on why security, data hygiene and resilience are one problem. The others: A data error at FNB became a privacy breach and Most AI data leaks run through approved tools.
Sources
- Inside the 24 hours that broke Microsoft, TechCentral, 16 November 2026, summarizing Microsoft’s preliminary post-incident report.
- The Next Web on Microsoft’s preliminary post-incident report, November 2026, citing The Register, Bloomberg, Politico and the AP.
- Microsoft hit by global outages: what happened and why?, Al Jazeera, 13 November 2026.
Linked sources are the record. Check them for the full reporting.