Massive AT&T outage also happened because of a bad update

If you're wondering if the recent global IT outage – caused by a bad update to cybersecurity company CrowdStrike's software which brought down millions of Windows PCs – is an isolated case, we can assure you that it is not.

In fact, something similar already happened earlier this year. A government investigation into a nationwide AT&T outage in February has shown that the cause was a bad network update, Ars Technica reported on Tuesday.

According to an FCC report, the outage "affected users in all 50 states," and "all voice and 5G data services for AT&T wireless customers were unavailable, affecting more than 125 million devices, blocking more than 92 million voice calls, and preventing more than 25,000 calls to 911 call centers." The FCC also noted that it took AT&T "at least 12 hours to fully restore service."

Mashable Light Speed Want more out-of-this world tech, space and science stories? Sign up for Mashable's weekly Light Speed newsletter. By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy. Thanks for signing up!

We've covered the outage as it happened, noting that it caused disruptions for users on other, unaffected networks, as they were unable to call AT&T customers. AT&T offered its customers a $5 dollar credit as apology.

SEE ALSO: How did one CrowdStrike mistake stop the world? We asked 3 experts.

The incident, according to the report, began "after AT&T implemented a network change with an equipment configuration error." But it wasn't just this one, isolated issue that made this outage so serious.

---

Related Stories

---

• Microsoft says EU rules made CrowdStrike outage possible
• Microsoft releases easier CrowdStrike fix
• CrowdStrike outage 3 days later: Where does everything stand now?
• CrowdStrike outage is still causing hundreds of flight cancellations daily
• Microsoft outage updates: CrowdStrike issue still wreaking havoc despite fix

The FCC Public Safety and Homeland Security Bureau analyzed the incident and found that the outage "was the result of several factors, all attributable to AT&T Mobility, including a configuration error, a lack of adherence to AT&T Mobility's internal procedures, a lack of peer review, a failure to adequately test after installation, inadequate laboratory testing, insufficient safeguards and controls to ensure approval of changes affecting the core network, a lack of controls to mitigate the effects of the outage once it began, and a variety of system issues that prolonged the outage once the configuration error had been remedied" (per Ars Technica).

SEE ALSO: It’s 4 days later and Delta is still affected by the CrowdStrike outage. Here’s why.

The story might not end there for AT&T, which is potentially facing a large fine. But it's another reminder that the global IT networks which we rely on are often more fragile than we think, and the safety procedures for critical systems need, in many case, some serious looking into.