KDnuggets Home » News » 2015 » Dec » Opinions, Interviews, Reports » 2016: The Year of Hadooplooza ( 16:n01 )

2016: The Year of Hadooplooza


Bruno Aziza examines the Hadoopalooza effect, how to avoid poor decisions to come back from the party a "Hadoop-loser", and what is needed to get value from data lakes.

By Bruno Aziza, CMO of AtScale.

HadoopaloozaI'm sure you're familiar with the term "Palooza": an all-out party where participants act like there's no tomorrow. Hadoop has been on everyone's lips and the tone about this technology has gone from neutral to positive in 2015 - to the extent that, in 2016, it's predicted that most enterprises will deploy Hadoop. This year Hadoop events might start to look a little bit like a "Palooza" - a fun all out event, where some decisions might be made and regretted shortly thereafter.

Don't fall for the "Hadoop-looza" extravaganza.

"Hadooplooza" versus "Hadoop-a-losers"

In April 2015, Barclays released a reporting showing increasing enterprise adoption of Hadoop. This last October, AtScale's Hadoop maturity survey (the industry's largest) revealed that 76% of those already using Hadoop were planning on doing more over the next 90 days and that, close than half of the companies that had not yet deployed Hadoop, would do so in the next 12 months.

Beware though. In 2016, new vendors will have you believe that you need to re-architect your entire data architecture to accomplish success with Hadoop. Others will ask to rip and replace the good work you've done with analytics' tools over the last decade. Don't believe any of that.

If you are using Tableau, Microsoft Excel or any other Business Intelligence tool for that matter, demand that your solution provider works with it, natively. Your team has spent way too much time, money and effort in getting these tools off the ground and adopted. It makes NO sense to replace them.

And understand the greatest value in Hadoop is no longer about cheap data storage. It's about business value and end-user adoption. The results of our study showed that the companies that gain the most value from Hadoop do so because they successfully connected existing BI investments to this next generation data platform. Companies that deploy Hadoop to drive revenues are close to 30% more likely to achieve value than those who are looking to cut costs.

Bottom line: Enjoy the Hadooplooza effect but don't make poor decisions and come back from the party, a Hadoop-loser.

data-lakeMismanaged Data Lakes: Where insights go to die

The term "Data Lakes" has become a popular term this last year. If you're not familiar with the term, a "Data Lake" often refers to the place companies choose to use as a repository to store the data pulled from their disparate systems.

Data Lakes are often used in the same context as Hadoop. Hadoop's ability to store any kind of data easily makes it ideal. There is one problem though. The companies that have used Hadoop, primarily as a Data Lake, without any plan or vision to make that data usable across the organization, are starting to realize that their lake is becoming a swamp.

Many have joked about the concept of "data gravity" and the fact that, as data goes unused, it tends to sink down and stagnate at the bottom of the lake. So much in fact, that companies forget what they have and your company's potential to create amazing insights die.

This problem is not to be taken lightly. The average organization spends $7.4M on Big Data analytics. Only 42% of decision-makers say they're satisfied with analytics at their companies, and less than half of the data available is being leveraged for insights. Data brings value when it's turned into insights. In this new year, challenge your team to think about Hadoop as a 'revenue generation' project, not a 'cost-saving' initiative.

Bottom line: Dumping Data into Hadoop and creating a Data Lake is step 1. If you don't create a vision that goes beyond cheap storage, your data will go unused and you will have nothing more than having created a data swamp.

Bio: Bruno Aziza is Chief Marketing Officer (CMO) for AtScale and is based in San Francisco.