|
Can Machine Learning on Big Data replace Domain Expertise? [256 votes total] | |
| Yes, it is only a question of time (89) |  35% |
| No, there are many domains where machine learning cannot beat domain expertise (142) | 55% |
| Not sure (25) | 10% |
Selected Comments
Gregory PS, Editor, When ML outscored Experts
Many such recent examples. At Strata 2012 ,
Claudia Perlich pointed out that she won data mining competitions on breast cancer, movie reviews, and customer behavior without any prior knowledge.
Look at results of most KDD Cups or Kaggle competitions - the winners are not experts in that particular domain
Dr Ravi Vadlamani, Machine Learning vs. Domain Experts
Whenever ML outperforms domain expertise, I would tend to conclude that domain expertise in that domain is either not complete or not completely utilized. Alternatively, it is possible that more knowledge is available in the data that was mined. Essentially, I believe and experienced that a suitable combination of knowledge-driven and data-driven techniques yields excellent results - better both of them in isolation. Dr Gregory, can you pls let us know in which competitions, ML outscored Domain expertise?
Ross Bettinger, ML replacing Human Domain Expertise
I am reminded of an early Isaac Asimov story in which a man is in competition with a Multivac computer (remember the EDVAC, and UNIVAC?). He is defeated in all contests of speed and knowledge (does IBM's Watson come to mind?).
At the end of the competition, he asks the computer (they had voice input a la Star Trek in Asimov's imaginary competition), "What are the magnitudes of a dream?" And all of the flashing lights on the computer's console began to wink out, one-by-one. And the computer was silent.
Perhaps I am being romantic to say that human intuition will always have a place in human life. But I believe that, no matter how thoroughly one or more domain experts are debriefed by knowledge engineers, there will still be "unknown unknowns" or emerging trends that will not be sensible to a computer program. Until a true AI comes along that can autonomously adapt to new and unanticipated experiences, I vote for the human element and say that ML will not triumph over domain expertise.
Vasudha Bhatnagar, Domain Experts are required
Model building is an exercise in Hypothesis Formulation. In some domains, hypothesis verification has to be supported by human intelligence, intuition and domain knowledge that is not available to the model building algorithm. This is particularly true in domains where the model exposes the symptoms, and assigning the causal factors has to undertaken by humans.
Eric King, Machine Learning vs. SME
In our experience, the best overall solution is achieved when a subject matter expert is included as a team member in the model development process -- mainly at the tails (business understanding, data understanding, -- then model evaluation, translation and deployment).
While less model oversight is required with advancements in machine learning, SME's should always play a role in evaluating process performance, lifetime model management, and ensuring that model results map to environmental realities.
John Garrett, Domain expertise
The question is not whether sophisticated data analysis improves decision making. Better data almost always improves decision making, and sophisticated data analysis should be helpful. The question is whether there is a role for human intuition to improve on the findings from sophisticated data analyses. People can always ask interesting questions for analysis, and, if the results are counterintuitive, further analyses to confirm or deny the conclusions are necessary. Someone has to make a decision on whether to believe the tool and act upon it, and that will require more expertise, not less.
Gregory PS, Editor, Can ML and Big Data beat domain expertise?
The results of recent analytics competitions show that in increasing number of domains machine learning and data mining experts are able to produce better results than domain experts. Witness the amazing progress in AI, computer game playing, machine translation, speech recognition. So, unless we think that humans have a magical ability that cannot be reproduced, eventually machines will be able to come up with better decisions. Sad, but probably inevitable.