Big Data Assessment – Key Business Drivers, Expected Benefits and Common Challenges

Recent survey on Big Data outlook reports increasing interest in Big Data for more accurate and timely decision-making; and concerns about project costs and ability to scale.

In the past few years we have seen a lot of action around Big Data. As the technology matures to handle such massive amount of data, market attention has gradually moved from the hype surrounding it to focus on deriving maximum business value. The rise of interest in Big Data across industries has been stunning, which can clearly be seen through the numerous vertical-focused Big Data conferences around the world and an increasing number of startups and Big Data projects within firms of all sizes.

Despite this immense interest and maturity growth, most companies are still figuring out their business use cases for Big Data,Quinstreet enterprise trying to understand the industry benchmarks so that they can set up their own specific business goals, and prepare themselves for some of the most common challenges faced by Big Data initiatives. To better understand the business drivers, expected benefits, and challenges of Big Data, QuinStreet Enterprise conducted a survey of 540 decision-makers involved in Big Data purchases.

Key highlights of Big Data assessment:
  • 77 percent of respondents consider Big Data analytics a priority
  • 72 percent cite enhancing the speed and accuracy of business decisions as a top benefit of Big Data analytics
  • 71 percent of mid-sized and large companies have plans for, or are currently involved with, Big Data initiatives

Though Big Data offers great opportunities, there are several challenges that need to be overcome first:
  • Big Data project and management costs
  • How to scale infrastructures to accommodate the large volumes of data to be analyzed
  • Overcoming data silos and application integration

The market has clearly evolved from the state of figuring out what to do with Big Data to considering it a top priority. This growing confidence towards Big Data can be contributed partially to the availability of powerful open source solutions such as Hadoop, which is a software framework for storage and processing of large datasets on clusters of commodity hardware. Tools such as Hadoop allow organizations to run Big Data analytics using commodity hardware, thus, lowering the cost significantly.

As we have seen in some recent KDnuggets interviews, successful Big Data initiatives are driven by well-planned business use cases. The survey found that the most popular business use cases for Big Data are increasing efficiencies and optimizing operations (improving speed/reducing complexity), improve customer retention, help with product development, and gain a competitive advantage. Business Use Cases Assessing the value of Big Data to organizations, the key benefits were recognized as improved and timely access to decision-making information, greater transparency, scalability and better change management.

However, since Big Data is still relatively new, significant challenges can be expected during implementation. The most common challenges cited during the survey include upgrading existing databases, making analytics easy for end users, the complexities of software and data integration, and data management (capturing, curation and storage).

Big Data is a major strain on most corporate IT infrastructures, with almost half (46%) of respondents saying that their company was already managing greater than 10 TB of data in a typical month. At such a scale, the cost of managing data for meaningful analysis can quickly grow out of control. Furthermore, the data volume is expected to continue growing at a rapid pace. The need for additional infrastructure solutions was evident in the survey results. Companies plan to leverage in-house hardware first and foremost. Data Volume for analytics per month
One of the critical success factors for a successful Big Data project is selecting the right tools, which has become increasingly challenging given the broad spectrum of open-source and proprietary solutions. About 44 percent said they were looking to use analytics databases. Following close behind at 41 percent were BI tools and relational databases. And more than a third of the respondents noted that they also use data warehouses, predictive analytics, data visualization tools, and discovery platforms.

The most interesting part of the survey came from the participants’ response regarding which vendors were they working with (or planned to work with) to provide their organization with Big Data analytics. The top answer (with 43% in support) was “none”. Only one vendor (IBM at 12 percent) got more than 10 percent of the votes. Thus, the market seems to be still evolving and companies are still not ready to fully invest into any particular Big Data solution.

From the survey it seems very clear that Big Data is no more merely a buzzword, it is a transformative technology trend with immense potential and can have real impact when implemented in alignment with the business strategy.

For further details on the survey and to download the executive brief, visit:
2014 Big Data Outlook: Big Data is Transformative – Where is Your Company?