The Force Awakens In Data – Industry Leaders Comment
While Rey, saw her force come to life in less then 30min, the data industry has been waiting for ‘that’ to happen, for half a decade. However, finally, business-focused analytics and data discovery are on the rise.
Automated insights
I normally start my courses at Harvard with the question, “Do we have this kind of hype in data? The answer is “Yes”, as it’s all about pattern recognition. Hidden patterns, in large datasets, help predict user behavior and help improve the value proposition of our products. Typically, a data scientist or an analyst will dig through the data, in order to surface patterns related to statistical relevant correlation or outliers.
This process can be automated to a good degree. This is where automated advanced analytics comes into play. Automated analytics is like a form of the ‘death star’ when it comes to the industry. With one stroke, a group of algorithms goes in parallel through the data, in order to detect correlations, clusters, outliers, anomalies, linkages, trends… you name it. It’s the brute force approach.
But correlation itself might not make an insight – let alone create an action – see the graph below showing the correlation between the divorce rate and the margarine consumption. Or as Gartner formulated it “that most business users do not have the training necessary to accurately conduct or interpret analysis.”
This is where BeyondCore gets involved. Arijit Sengupta, founder of BeyondCore, build this platform not only to surface all kinds of correlations, but also to warn the business user about potential hidden factors, in an effort to protect the user from statistically unsound decisions.
“Most business users see a pretty graph and think they can take action based on it. In reality they regularly take actions based on misleading or misunderstood graphs. Automated Analysis software need to guide us to statistically-sound actionable insights and explain the patterns in detail so that business users can act with confidence.” (Click to Tweet) (Arijit Sengupta)
Visualization
We know all that a picture says more than 1000 words. Thus, a data platform needs to be visual and allow the business user to showcase the most important insight. With the onset of HighCharts, we have seen many companies try to out-bid their competitors, by using their superior number of chart types. But be aware – even without actionable insights, one can create good visualization. We call this, “beautiful but useless”. As the author of the book, “Data Mining for Dummies”, Meta Brown rightly said:
“Tools are just… tools. They should not define your work. It’s your job to understand the problem, identify goals and choose the tools and processes that help you reach those goals as easily as possible.” (Click to Tweet) (@metabrown312)
Outside world
More power leads to more publicity. In the past, the BI team and their insights tugged away in the basements of companies. But now, data has become a first class citizen within those companies. It is thus no surprise that it has become important to communicate insights to the outside world. Only insights that are seen can be acted on. Thus the new set of data platforms make it easy to publish their findings to an internal audience as well as embedded those insights into products and services to their customers. As Chris Wintermeyer told me lately at a dinner table:
“Much of the success of any data platform will hinge on the way that, the insights generated, are shared and discussed.” (Click to Tweet) (@ChrisAtDomo)
The future
With the business force awakening, the future seems bright. Most companies, by now, have the right vision. That’s not really hard, since we have talked about the data needed ‘now’, for at least half a decade. However, the Gartner magic quadrant does not list any ‘challenger’. Is this the end of innovation in the data space?
Maybe. But maybe the true challenges today are no longer within technology, as such, but in a balance to use the insights to support our decisions and not to determine our actions blindly. As the Netflix CEO, Reed Hastings, recently pointed out: Data has a support function (Click to Tweet) (@reedhastings)
For humans, with or without the force, that rule is still true: “actionable your data must be”. (Click to Tweet)
This article was originally published on my Forbes Blog.
Bio: Lutz Finger is Director Analytics at LinkedIn, was Data Scientist in Residence at Cornell, Lecturer at Harvard. He is a published Author and Business Angel, based in San Francisco Bay area.
Related:
