Berkeley Michael Jordan says Big Data is transformative, not a delusion

By Gregory Piatetsky, @kdnuggets, Nov 7, 2014.

Recent interview in IEEE Spectrum of Berkeley Prof. and noted Machine Learning expert Michael Jordan has caused a lot of stir, especially given its very provocative title "The Delusions of Big Data and Other Huge Engineering Efforts".

Much less publicity was given to Michael Jordan reaction to the interview title (thanks to Usama Fayyad @usamaf for pointing me to this article).

In his blog Big Data, Hype, the Media and Other Provocative Words to Put in a Title Michael Jordan writes that the title was placed without his knowledge on the interview.

He says

I don't feel that Big Data is a delusion at all; rather, it's a transformative topic, one that is changing academia (e.g., for the first time in my 25-year career, a topic has emerged that almost everyone in academia feels is on the critical path for their sub-discipline), and is changing society (most notably, the micro-economies made possible by learning about individual preferences and then connecting suppliers and consumers directly are transformative).

He adds

it's a *major engineering and mathematical challenge*, one that will not be solved by just gluing together a few existing ideas from statistics, optimization, databases and computer systems.

His point was that Big Data is a Huge Engineering Effort and that that's no Delusion, so you can imagine his dismay at a title that said exactly the opposite.

However, none of this changes his warning about

Big Data Winter ahead - unless we change course,

and think more carefully about what we find in Big Data.

A good start for avoiding Big Data winter is to understand that every prediction has error bars. If error bars are not shown, it probably means they are uncomfortably large and should reduce your trust in the prediction.