The reality, in my opinion, is that the ambiguity of the term and lack of tangible value from investments ascribed to it led to its eventual downfall. The rationale was presumably because “big data become a part of many hype cycles,” according to Datanami. With respect to big data, it followed this curve but then, interestingly, dropped altogether before reaching some generally agreed upon level of acceptance. This scenario plays out repeatedly and is particularly relevant in analytics.
Expectations are inflated, which ultimately leads to a general letdown (Trough of Disillusionment) and then (hopefully) a reasonable level of common understanding whereby the technology matures and finds its stated value in the market (Plateau of Productivity). Gartner has a popular concept called the “ Hype Cycle.” In this cycle (see graph below) there’s an initial rush toward a new technology or concept. There’s an old saying, “The definition of insanity is doing the same thing over and over and expecting different results.” We see this play out time and time again in the world of analytics. Instead, a great deal of opportunity was lost chasing the hype of big data. Whether or not people were directly involved in analytics, they owed it to themselves and their respective organizations to delve deeper into what was behind this concept and how it could be applied. The analytical evolution was certainly underway and creating a monumental shift in thinking, but people became fixated on a nebulous term and not pragmatic results. Unfortunately, what it typically didn’t have was anything to do with how the innovative use of big data and analytics would make your company better. It was a certain set of analytical methods and tools. So, what happened to it? In my opinion, the problem is that it’s a term that meant everything and nothing. Now, several years later, after a lot of time and money has been invested, what strikes me is that I rarely hear the term “big data” anymore. At the end of the day, however, I was left with more questions about big data than when I started. Follow up presentations discussed a myriad of analytical opportunities (some, incidentally, were about using existing techniques and technologies). People in the audience then broke into workshops to figure out how they would leverage big data.
And the fourth talked about analyzing audio files (without discussing why that was relevant for this retailer).Īll the speakers talked glowingly about their topics, the exciting future, and the fact that people should sit up and take notice. Another talked about general data warehousing concepts that had been around for several years. Another talked about open-source technologies (primarily Hadoop). One speaker talked about the “Four Vs” (I’ll let you look that up and decipher it for me because I never really understood the positioning of that definition). To my surprise, not one of the speakers made a cogent point about what big data was and why the audience should care about it. Big data was a relatively new concept back then and the workshop included four speakers who explained how it would transform that company’s business. Seven years ago, I was involved in a corporate workshop headed by a customer’s CIO to discuss that company’s big data strategy.