Blue Screen of Death workaround revealed

The company behind a faulty update, which triggered a chaotic global IT outage, has revealed a workaround to the dreaded “Blue Screen of Death” leaving computers stuck on a restart loop.

Computer systems across Australia crashed about 3pm on Friday after a faulty driver update was pushed out by American cybersecurity giant CrowdStrike.

Entire Microsoft systems were wiped out as a result.

Office workers and customer-facing screens in places like supermarkets and airports were left facing the the “Blue Screen of Death” (BSOD) – causing computers to repeatedly reboot and crash.

Airlines, banks, media companies, petrol stations and other major businesses and retailers all came to a grinding halt as a result.

Mac and Linux hosts are not impacted.

In a statement, CrowdStrike said it was actively working with customers impacted by the defect in their latest driver update.

They confirmed the outage was not caused by a malicious hack.

“The issue has been identified, isolated and a fix has been deployed,” the company said on Friday night.

“We further recommend organisations ensure they’re communicating with CrowdStrike representatives through official channels.”
The company also published a slightly technical workaround for computers experiencing the BSOD:

CrowdStrike’s workaround for BSOD computers

• Reboot the host to give it an opportunity to download the reverted channel file. We strongly recommend putting the host on a wired network (as opposed to Wi-Fi) prior to rebooting as the host will acquire internet connectivity considerably faster via ethernet.
• If the host crashes again, then:
  • Boot Windows into Safe Mode or the Windows Recovery Environment
    • NOTE: Putting the host on a wired network (as opposed to Wi-Fi) and using Safe Mode with Networking can help remediation.
  • Navigate to the %WINDIR%System32driversCrowdStrike directory
  • Windows Recovery defaults to X:windowssystem32
    • Navigate to the appropriate partition first (default is C:), and navigate to the crowdstrike directory:
      • C:
      • cd windowssystem32driverscrowdstrike
    • Note: On WinRE/WinPE, navigate to the WindowsSystem32driversCrowdStrike directory of the OS volume
  • Locate the file matching “C-00000291*. sys” and delete it.
    • Do not delete or change any other files or folders
  • Cold Boot the host
    • Shutdown the host.
    • Start host from the off state.
• Note: BitLocker-encrypted hosts may require a recovery key.

CrowdStrike boss apologises for outage

Speaking to the American TV program Today, CrowdStrike’s chief executive George Kurtz said he was “deeply sorry” for the outage.

“Essentially … the system was sent an update, that update had a software bug in it and caused an issue with the Microsoft operating system,” Mr Kurtz told the program.

“We identified this very quickly and remediated the issue.”

Mr Kurtz clarified the incident was not a cyber attack and was solely confined to the faulty update.

“It could be some time for some systems that won’t just automatically recover,” he said.

Some systems have returned to normal but airports were still being plagued by cancelled flights as a result of the outage.

At Sydney airport, multiple planes were grounded over Friday and Saturday from a combination of the IT issues and high winds.

Check-outs at Woolworths and Coles were also rocked by the BSOD.

Home Affairs Minister Clare O’Neil urged Australians to watch out for scammers in the wake of the tech collapse, amid reports emails purporting to be from CrowdStrike were asking people for their bank details.

“Don’t put in any details. If someone has called you and is suggesting that they’re going to help you talk you through a reboot of your system, I would hang up the phone,” she said.

“Then just step back and think. Have a look at the communication that you’ve just received and just ask, ‘Does it make sense for you?’ Your bank is not going to ask you to put your bank details in.

“If you have given away some personal information, just make sure that you’re contacting your banking institution, for example, and making sure that you let them know that you’re concerned about a phone call or an email that you might have responded to.”