Big Data Myth and Big Data Velocity
"There is only so much static data in the world as of today. The vast majority of new data is arriving from a high velocity source", says VoltDB CTO Scott Jarr in an interview with Roberto Zicari.
Roberto Zicari, the editor of ODBMS.org, recently interviewed Scott Jarr, Co-founder and Chief Strategy Officer of VoltDB, (co-founded by a prominent database scientist and enterpreneur Michael Stonebraker). The focus of the interview was on one of the key challenges of Big Data: Velocity. Here is an excerpt. Gregory PS.
RVZ: Q1. Marc Geall, past Head of European Technology Research at Deutsche Bank AG/London, writes about the "Big Data myth", claiming that there is:
1) limited need of petabyte-scale data today,
2) very low proportion of databases in corporate deployment which requires more than tens of TB of data to be handled, and
3) lack of availability and high cost of highly skilled operators (often post-doctoral) to operate highly scalable NoSQL clusters.
What is your take on this?
Scott Jarr: Interestingly I agree with a lot of this for today. However, I also believe we are in the midst of a massive shift in business to what I call data-as-a-priority.
We are just beginning, but you can already see the signs. People are loathed to get rid of anything, sensors are capturing finer resolutions, and people want to make far more, data informed decisions.
I also believe that the value that corporate IT teams were able to extract from data with the advent of data warehouses really whet the appetite of what could be done with data. We are now seeing people ask questions like "why can't I see this faster," or "how do we use this incoming data to better serve customers," or "how can we beat the other guys with our data."
Data is becoming viewed as a corporate weapon. Add inbound data rates (velocity) combined with the desire to use data for better decisions and you have data sizes that will dwarf what is considered typical today. And almost no industry is excluded. The cost ceiling has collapsed.