From Data Mining to Big Data and Beyond

I look at the changing trends in analyzing data and the popularity of different terms, such as statistics, data mining, knowledge discovery, data science, and Big Data, and examine the gap between marketing and popular perceptions.

Date:

by Gregory Piatetsky-Shapiro, April 18, 2012.

As part of my webcast/briefing with Joydeep Das of SAP, I wrote the following article

This article looks at the changing trends in analyzing data and the popularity of different terms, such as statistics, data mining, knowledge discovery in data (KDD), predictive analytics, data science, and Big Data, and examines the gap between marketing and popular perceptions of these terms.

Learning from observations is a very deep-seated human trait. Our ancestors who knew how to avoid predators and search for food had better chances for survival and left us with an instinct for finding patterns in data. However, the methods that worked well for a small number of lions are not adequate for making inferences from a larger number of observations, and our intuition is not well-suited for statistical inferences - consider various superstitions, the popularity of astrology, or a lucky streak in gambling, etc.

...

Data Mining and Analytics Trends, 2004-2011

Note that after at the end of 2005 the searches for "Analytics" spiked dramatically, about the same time as Google introduced Google Analytics. Also note an increase around summer of 2007, which coincides with the release of Competing on Analytics: The New Science of Winning book, by Thomas H. Davenport and Jeanne G. Harris (March 2007). Other interesting features include sharp drops around December vacation time - even dedicated analysts need some time off!

However, "Analytics" is a very broad term and there are many kinds of analytics. Looking deeper, I found that about 52% of all searches for "analytics" were for "google analytics".

Excluding "Google" from analytics, we see that there are now about 2.75 times more searches for various kinds of "analytics" than for "data mining". However, "data mining" mentions dominate the news reference volume. This is an interesting example of a search/news gap - between use of the term in news articles and its popular use.

Data Mining and Analytics Search and News Trends, 2004-2011

Read the entire article at www.insideanalysis.com/2012/04/data-mining-and-beyond/