CrowdStrike has identified a bug in its own test software as the cause of last week’s mass-crash event affecting 8.5 million Windows systems.
In an update to its remediation guide, CrowdStrike included a Preliminary Post Incident Review (PIR) explaining how its Falcon Sensor, which ships with “Sensor Content”, and receives updates via “Rapid Response Content”, failed.
The Sensor Content updates use “Template Types” for threat detection, which are implemented as “Template Instances”. These instances map specific behaviors for the sensor software to monitor and prevent.
In February 2024, CrowdStrike introduced a new “InterProcessCommunication (IPC) Template Type” designed to detect attacks using Named Pipes. This Template Type passed testing and was deployed in March, followed by additional instances in April. However, on July 19, an instance containing “problematic content data” was released due to a bug in the “Content Validator”.
This bug allowed the flawed instance into production, leading to an “out-of-bounds memory read triggering an exception”, which caused Windows systems to crash.
Brief explanation of why #crowdstrike update caused Global IT Outage yesterday.
Crowdstrikes update to their Falcon software had a bug that caused #windows machines to #BSOD, only can be fixed manually.
Hoping they can recover from this and that the millions affected return… pic.twitter.com/P0grMmafz4
— Abhishek Sastri (@saasabhi) July 21, 2024
CrowdStrike has promised to enhance testing, stagger releases, offer users more control over deployments, and provide detailed release notes. A full root cause analysis will be released once the investigation is complete.
As the tech community rebuilds, it’s clear that rigorous testing and transparency are crucial in preventing such widespread issues.
Start Growing with Cloudways Today.
Our Clients Love us because we never compromise on these
Abdul Rehman
Abdul is a tech-savvy, coffee-fueled, and creatively driven marketer who loves keeping up with the latest software updates and tech gadgets. He’s also a skilled technical writer who can explain complex concepts simply for a broad audience. Abdul enjoys sharing his knowledge of the Cloud industry through user manuals, documentation, and blog posts.