Date:
The previous KDnuggets Poll asked:
Can Machine Learning on Big Data replace Domain Expertise?
The results in the table below suggest that data miners and analytics professionals who read KDnuggets are skeptical that Machine Learning on Big Data can replace Domain Expertise.
However, there are growing number examples where Machine Learning + Big Data outperform domain expertise. At Strata 2012 , Claudia Perlich pointed out that she won data mining competitions on breast cancer, movie reviews, and customer behavior without any prior knowledge.
Look also at results of most KDD Cups or recent Kaggle competitions - the winners are not experts in that particular domain.
|
Can Machine Learning on Big Data replace Domain Expertise? | |
| Yes, it is only a question of time (89) |  35% |
| No, there are many domains where machine learning cannot beat domain expertise (142) | 55% |
| Not sure (25) | 10% |
Selected Comments
Ross Bettinger, ML replacing Human Domain ExpertiseHere are full results and comments of KDnuggets 2012 Poll:
I am reminded of an early Isaac Asimov story in which a man is in competition with a Multivac computer (remember the EDVAC, and UNIVAC?). He is defeated in all contests of speed and knowledge (does IBM's Watson come to mind?).At the end of the competition, he asks the computer (they had voice input a la Star Trek in Asimov's imaginary competition), "What are the magnitudes of a dream?" And all of the flashing lights on the computer's console began to wink out, one-by-one. And the computer was silent.
Perhaps I am being romantic to say that human intuition will always have a place in human life. But I believe that, no matter how thoroughly one or more domain experts are debriefed by knowledge engineers, there will still be "unknown unknowns" or emerging trends that will not be sensible to a computer program. Until a true AI comes along that can autonomously adapt to new and unanticipated experiences, I vote for the human element and say that ML will not triumph over domain expertise.
Eric King, Machine Learning vs. SME
In our experience, the best overall solution is achieved when a subject matter expert is included as a team member in the model development process -- mainly at the tails (business understanding, data understanding, -- then model evaluation, translation and deployment).While less model oversight is required with advancements in machine learning, SME's should always play a role in evaluating process performance, lifetime model management, and ensuring that model results map to environmental realities.
Can Machine Learning on Big Data replace Domain Expertise?