Google loses data after successive lightning strikes in Belgium

However, Google says some of the data that was lost to the lightning strikes were located in storage systems that were more susceptible to power failure because of repeated or extended battery drains. Google took full responsibility for the loss of data and apologized to its customers for the “exceptional” event, reports NBC.

Earlier it was thought that the electrical storm had struck the mighty Google data centre but it was later clarified that it was the local grid facility that had been affected. In a State of Iowa data center last year, a power surge or another electrical phenomenon caused a transient voltage surge suppression box to fail, which in turn cut power to the whole data center. Google says these both flipped on as expected to prevent damage to the disks.

Google said it was “wholly responsible” for the outage, and it urged affected customers to consider duplicating and storing their data through other services that it provides. Data centers are built with lightning rods to dissipate strikes, but sometimes lightning does get through.

Google must have done something to anger the gods, for they have blasted one of Google’s European data centers with lightning not once, not twice, not thrice, but four times.

In its incident report, Google says that less than 0.000001% of its Western Europe permanent disk space was lost, but because the data was replicated across multiple servers, customers did not permanently lose data.

About five per cent of disks in the data centre recorded “at least one I/O read or write failure” during the incident. In most cases, the “affected disks sporadically returned I/O errors to their attached GCE instances, and also typically returned errors for management operations such as snapshot creation”. Automatic backup systems quickly restored power, and the majority of the data was recovered, but a miniscule fraction was reportedly permanently affected.

Google’s infrastructure team is in the process of replacing its storage systems with newer hardware that’s less susceptible…. The company adds that it’s conducted a review of the incident and “Several opportunities have been identified to increase physical and procedural resilience”.

James Wilman of Future-Tech said that although lightning strikes can be coped with by the centres, it is not impossible for strikes to get through defences.