Google Labels the Internet Evil
Okay, it was definitely an accident. And it was fixed very quickly. But still, Google’s recent error in marking every site on the whole Internet as “potentially malicious” shows how human error can cause a major technical faux pas.
The error itself involved the accidental addition of what’s known as a “wildcard” character to a file used to identify suspect websites. While this file was being updated, “the URL of ‘/’ was mistakenly checked in as a value to the file, and ‘/’ expands to all URLs.” This is a very simple mistake, and probably just involved finger failure on the part of the administrator. It was certainly not malicious on Google’s part, but was certainly very embarrassing.
As a knock-on effect, the error also may have caused some email to be mis-classified. According to Google’s statement “the block list is also used in its spam filters, so legitimate messages may have been classified as spam.” They’re now working on ways to correctly re-classify any mail that was incorrectly labeled, but this is only a retrospective correction since those initial messages were almost certainly discarded.
This isn’t the first time administrator error has caused outages or other problems across the Internet, and it won’t be the last. The complexity of the systems and processes involved increases all the time. It’s also obvious that such outages will receive increased media attention. The overall importance of the ‘Net increases daily as more people and individuals turn to it as a work-saving and communications-enhancing tool.
According to my own Second Law of Computing, an increase in the importance of a given system mandates a parallel increase in complexity. You can’t run a “five nines” operation without putting serious safeguards and massive redundancy into place. Redundancy makes the system harder to manage. And as “Scotty” from Star Trek once said, “the more they overtake the plumbing, the easier it is to stop up the drain.”
The simplest error can take whole systems down. It doesn’t matter whether a construction backhoe cuts through your fiber-optiic cable or a janitor accidentally unplugs a critical router — the results are the same.