BlueOnyx
CloudInfrastructureDevOpsResilienceObservability

The GitHub Outage and the Blind Spot That Took Everything Down

Théodore BaillyPublished on 22 août 20265 min read
Rétroviseur de voiture reflétant un coucher de soleil

Introduction

On August 17, 2026, GitHub went dark for seven hours and forty-seven minutes. Authentication, APIs, Actions, Copilot, pull requests, issues — the entire platform collapsed simultaneously, locking millions of developers out of their primary tool. It wasn't even the first incident that month: GitHub had already suffered an outage of more than seven hours on Actions on August 6. But the scale of the August 17 event — and the exceptional precision of the postmortem published three days later — make it a defining case study in the fragility of modern cloud infrastructure.

A Cascade, Not a Bug

The root cause was neither a failed deployment, nor a software vulnerability, nor a cyberattack. It was a capacity failure triggered by an unprecedented traffic spike.

At the center of the problem: an Istio sidecar — the component responsible for mediating service-to-service communication in a service mesh environment — hit its concurrent processing limit inside GitHub's US data center. Critically, the auto-scaling mechanism was monitoring the health of the primary service, not the sidecar. No alert fired. No scale-out was triggered. Four HAProxy servers, the platform's internal load balancers, absorbed the overflow until they saturated, blocking authentication flows and setting off a chain reaction across every downstream service.

A second mechanism dramatically extended the incident: a latent bug in the VS Code Copilot client caused it to retry failed API calls without any cap. That behavior multiplied request volume tenfold — from a baseline of 7,000 to 9,000 requests per second to over 100,000 during the crisis. This retry storm, a well-documented failure mode in distributed architectures, prevented any stabilization attempt for hours.

The Growth That Creates Fragility

Context matters here: monthly commits processed by GitHub grew from 1.4 billion in April to 2.9 billion in August — a doubling in just four months. That growth is largely driven by the adoption of AI-assisted coding and automation tools, which generate commit volumes with no historical precedent in manual development workflows.

This pace fundamentally changes capacity planning. Infrastructure sized six months ago can become dangerously undersized long before the next annual review. And when growth curves accelerate non-linearly, observability gaps — such as an auto-scaling policy that doesn't cover every component in a critical processing chain — become structural vulnerabilities, not edge cases.

The Remediation Roadmap

In response to the incident, GitHub announced a concrete set of corrective measures: adding more than 3 million CPU cores and 120 petabytes of storage, accelerating migration to Azure infrastructure — which now handles approximately 58% of platform load — tightening retry policies across critical services, and strengthening component isolation to prevent localized failures from cascading system-wide.

What This Means for Infrastructure Teams

The incident surfaces an uncomfortable truth: in modern service mesh architectures, partial observability is as dangerous as no observability at all. Monitoring a service without monitoring its direct dependencies — sidecars, proxies, load balancers — creates blind spots that only reveal themselves under extreme conditions.

For teams running critical infrastructure, the takeaways are actionable: auto-scaling configurations must cover the entire processing chain, not just primary services; retry policies require regular auditing with hard caps enforced at the client level; and capacity plans must account for non-linear growth scenarios. What this postmortem documents with rare precision is the anatomy of an outage that many organizations — well beyond GitHub — are structurally positioned to replicate.

Share

The GitHub Outage and the Blind Spot That Took Everything Down