Google Flu Trends

In 2008, Google launched Google Flu Trends (GFT) to predict the spread of influenza across the US. The service was able to predict flu outbreaks one to two weeks faster than the CDC whose numbers were based on reports from doctors.

Google did this not by talking to patients or doctors but by determining the spread from the frequency and location of flu-related search terms. The likelihood of outbreak would be higher in states where flu-related searches were higher.

A fairy tale beginning

The ability to quickly detect outbreaks was welcomed by all, and Google’s algorithms were also found to have delivered accurate results.

The CDC had worked closely with Google on the project and was excited about the use of this technology. In their own words, “In theory at least, this idea can be used for any disease and any health problem”.

Other than an issue in 2009, where GFT was unable to detect swine flu and for which the algorithm was later updated, all seemed to be going well.

GFT became the poster child for big data and analytics. Big data cheerleaders hailed how scientific research and statistical models were now obsolete. All you needed was to have enough data and then find patterns in it.

The downfall

In 2013, GFT was in the news again, once more thanks to an outbreak. However, this time around, GFT was an unexpected victim. A comparison revealed that GFT had over overestimated flu levels for the 2012-2013 seasons by almost twice as that determined by the CDC.

This was only the beginning. In an article published in Science in 2014, a team of 4 Harvard-affiliated scientists showed that GFT had been overestimating flu levels for a much longer period of time and had been missing high, including “100 out of 108 weeks starting with August 2011.” They talked about the lack of transparency in Google’s algorithms and the fact that they were looking only at the numbers and not the context.

The media was swift in taking Google GFT to task and did not hold any punches back. It was now a case study for big data failure. You can find some of these responses here,here and here.

Why GFT was not a failure

The media was right in highlighting the loopholes with the thinking around big data, but terming GFT as a failure was far off the mark.

Right from the start, the GFT team had maintained that it was not a replacement but instead a complement to existing scientific methods. In an article in Nature in 2009, the Google and CDC teams wrote how the system could be used to signal an outbreak at the earliest –

“The early detection provided by this approach may become an important line of defense against future influenza epidemics in the United States, and perhaps eventually in international settings“

They further added – “This system is not designed to be a replacement for traditional surveillance networks or supplant the need for laboratory-based diagnoses and surveillance.”

They were not trying to figure out causation i.e. what caused what. Rather they were just identifying patterns in the data to detect correlation. Unfortunately, over-excited big data enthusiasts made it more than it was supposed to be and the product became a victim of its own success.

A valuable lesson

In summary, GFT did achieve what it was designed for. Yes, it was not the most accurate, but that was never really its purpose. The service has been discontinued since and Google would, in all probability, come up with something more robust and stronger than before but the whole episode has hopefully taught us an important lesson.

Big data is indeed revolutionizing the world. It is helping us discover new things that we wouldn’t normally be able to do. But it does not solve every problem. And when it does, we should realize that we might not really know why it happened.

Causation is, often times, very important. If our understanding of the matter is limited, we may not be prepared for any unexpected events or anomalies. If the problem’s severity combined with its recurrence justifies doing so, then more research and study should be called for.

About MResult

MResult helps companies build and manage technology solutions around their missions, goals and people.
Learn more here.