In a preliminary incident report dated November 15, Microsoft explained what triggered the major Microsoft 365 outage and how the damage spread across multiple regions. On the network gateway that accepts connections to each service, Teams traffic had been switched to the primary gateway, which Exchange Online also uses, changing both the load and the mix of requests. When servers began to stop responding during the peak US usage window, the mechanism that redirects traffic to servers still running made the failure worse. The report defines the outage period as 23 hours 50 minutes, from 1:20 on November 13 to 1:10 on November 14 (Japan time). The impact varied by region, connection method and time of day.

AD

15 hours 13 minutes from the Teams switch to the outage

According to the preliminary incident report prepared by Microsoft (PDF, public mirror), 15 hours and 13 minutes passed between the moment Teams traffic moved to the primary gateway and the official start of the outage. Response records show the automatic Teams switch took place at 10:07 on November 12 (Japan time; 01:07 UTC the same day). The official outage start was set at 1:20 on November 13 (16:20 UTC on November 12), when a high-priority alert signaling impact on Exchange Online fired. During the intervening hours, load on the primary gateway kept rising gradually, and in some regions server failures were already occurring sporadically before the alert, triggering automatic recovery processes.

Microsoft 365 has two independent gateways with different configurations. The design allows service traffic to be moved between them during a failure, so Teams shifting from the alternate gateway to the primary was itself part of normal operations. However, the inflow of Teams traffic increased load on the primary and changed the mix of requests it handled. On top of that, about 14% of the primary gateway's server processing capacity had been unavailable since the previous day because of a separate problem. The remaining capacity had been able to absorb the load until then, but it ran short when US usage peaked.

The direct cause was a lock contention bug in a Windows internal process that handles connections. Multiple operations tried to acquire the same exclusive lock and ended up waiting, and under certain load conditions this consumed all CPU and memory. Requests then piled up and the servers stopped responding. The code containing the problem was specific to the Microsoft 365 gateway and had been rolled out gradually over several weeks starting in July 2026. The flaw did not surface for more than three months afterward.

According to Microsoft, traffic composition and server hardware configurations differed between the test and production environments. Load testing mainly used lower-spec machines with fewer CPU cores, but the lock contention turned out to be more severe on newer machines with higher core counts. Even with load testing in place, differences in the combination of concurrent requests or in hardware configuration mean the same contention does not necessarily occur as it does in production.

Redistributing traffic to protect servers made the failure cascade

At 1:03 on November 13, Microsoft's monitoring systems detected a rise in connection anomalies on the user side. At 1:17, the system began automatically redirecting traffic from unresponsive servers to healthy ones. Normally, this mechanism keeps connections to the service intact even if some servers go down. This time, however, the servers that took on the traffic hit the same bug once their load rose, and traffic concentrated on the remaining servers in a vicious cycle.

At 4:20, the operations team concluded that the overload was caused by a shrinking pool of working servers and traffic concentrating on those that remained. The cascade spread in under two hours, and at its worst more than 80% of servers on the affected gateway stopped responding. The roughly 14% lost before the outage refers to processing capacity, whereas the figure above 80% refers to the share of servers that stopped responding because of the cascade. Google's SRE documentation likewise describes cascading failures in which traffic from failed servers is redirected to healthy ones, the recipients become overloaded, and the number of servers able to operate falls further.

The gateway in question manages thousands of cells, each with its own hardware, network resources and processing capacity, grouped by region. With the isolation design explained in the Azure Architecture Center (Bulkhead pattern), dividing resources into multiple groups can limit the impact on other cells when one fails. In this case, however, Microsoft says a separate problem prevented adequate isolation between cells, and the server failures spread across regions.

AD

Why Exchange Online could not be moved to another path

Teams, OneDrive, SharePoint, Office apps and other services were able to recover some functions early by moving their traffic away from the problematic gateway. Teams chat and meetings were available for most of the outage period. Exchange Online, by contrast, is a very large service and could not be switched in the same way, Microsoft said. Many users lost access to their mailboxes through multiple connection methods, including Outlook and the web client, and mail delivery was also delayed. Features that depend on Exchange Online in Teams, Copilot and other services were affected as well.

By 0:05 on November 14, near the end of the outage, Microsoft had finished moving traffic from customers who met the switching conditions to the alternate path. This further improved sign-in and access to services in the affected regions. However, stably restoring Exchange Online across the whole service required fixing the gateway's connection handling and bringing back servers capable of processing traffic.

The outage also disrupted day-to-day work at organizations using Microsoft 365. According to a report from the UK's NHS Supply Chain, warehouses and transport kept running as normal, but new orders and some delivery notifications were affected. In home delivery services, receipt of patient referrals was also delayed. The organization directed suppliers who receive orders by email to retrieve purchase orders from its portal, and said it would use the telephone to contact customers and suppliers where necessary. An update on the afternoon of November 13 (UK time) said external messages had begun to arrive but a backlog might remain; on November 16 it reported that external messages were being received normally again.

Restarted servers failed again; recovery required a code fix

At 5:00 on November 13, Microsoft bypassed a component believed to be causing problems with the health check that determines whether a server can be used as a traffic destination. At 6:00 it switched some servers to a different method and temporarily returned the affected servers to service. By 7:00, however, monitoring data showed that these measures had not led to overall recovery. At 8:55, Microsoft confirmed that servers behaved normally right after coming back, but stopped responding again once they began accepting traffic. Servers that could respond immediately after startup exhausted their processing capacity when they took on the backlog of queued traffic.

The investigation team traced the problem to lock contention in the process that establishes connections. At 10:53, Microsoft halted part of the automatic traffic redistribution and took measures such as throttling traffic flow, which improved conditions in some regions but did not achieve stable recovery. The normal automatic recovery mechanisms had not anticipated so many servers becoming unresponsive at once. And because the change responsible had been deployed several months earlier, standard rollbacks, staged rollouts and feature-flag toggles could not resolve the problem quickly.

Ultimately, Microsoft created a code update that restored connection handling to its previous, healthy behavior. It was applied to some servers at 16:20 on November 13, and after stable operation was confirmed at 17:00, the rollout was widened, prioritizing regions entering business hours and others. Target server groups were taken out of service, updated and restarted, then returned to accepting traffic. At 1:10 on November 14, Microsoft confirmed that direct user impact had been resolved, and it continued to monitor operations while working through the backlog of queued mail. The final customer-facing updates for incident IDs EX1550543 and MO1550821 were published at 13:15 the same day.

AD

Final report due November 20 will address isolation between regions

Microsoft extended mail delivery grace periods so that delayed messages would not be bounced as undeliverable. It also added processing capacity to support recovery and strengthened monitoring of traffic redistribution and connection handling. In addition, it placed limits on redistribution to keep traffic routing from triggering cascading failures.

The preliminary report does not explain why isolation between cells failed to stop the cascade, or how that relates to the capacity shortfall that had persisted since the previous day. It also does not disclose the conditions for routing traffic, the load level at which the lock contention becomes apparent, or per-region server counts and user numbers. A final report, including details of the corrective actions, is scheduled for publication on November 20. A key test of the isolation design will be whether, when production traffic composition is reproduced on high-core-count servers, the destination servers keep processing and the impact stays contained within a cell even if some cells fail.