Are Most Machine Learning Experts Turning to Deep Learning?

Read a short opinion on what the impact of machine learning researchers focusing on deep learning will be.

By Zeeshan Zia, Computer Vision and Machine Learning Researcher, Microsoft.

Hypnotic sky

Yes, most faculty, graduate students, and a lot of engineering teams in industry have already abandoned everything else and shifted to deep learning. Most new graduate students in applied areas such as computer vision that I meet, know nothing about probabilistic graphical models for instance, and their proposed solution to any problem is a CNN/LSTM/GAN.

The following is a blog post style paper by the leading pattern recognition professor Rama Chellappa, and one of the biggest authorities on face and biometrics, where he laments this state of affairs:

The changing fortunes of pattern recognition and computer vision

It is a huge deal to have an algorithm that can absorb large amounts of data - which is what deep learning methods enable. The (re-)discovery of such an algorithm (deep neural network training) has thus made possible many new applications, which were not possible just a few years ago. Thus the excitement around deep learning isn’t surprising at all. However, this excitement is bound to fade as the number-of-fresh-applications-enabled-per-new-quarter goes down. How excited are you about the steam engine today?

There are other important problems in machine learning. It is very desirable to be able to inject domain knowledge into machine learning models, which is something that deep learning methods aren't able to do. We already know quite a bit about English language grammar and sentence construction, why is it then that our latest and greatest deep learning based language model can’t be guaranteed to obey those rules? Why does it have to learn every damn thing from data only? Similarly, if we are really going to replace human decision makers by machines, we need algorithms that can perform reflective thinking. Like Marvin Minsky used to quip, good luck getting a DNN to reason that you can pull a string, but not push it! Another example of an open problem is interpretability. If you are unleashing a DNN to make lending decisions (by a bank), you had better be sure its not learning to be a racist!

As economic incentives fade, and research leaders make breakthroughs on problems for which deep learning alone isn’t particularly amenable, interest in other techniques, perhaps that can complement deep learning methods will return. I don’t see deep learning as completely over-shadowing machine learning five years down the road.

Bio: Zeeshan Zia researches computer vision and machine learning solutions at Microsoft. In the past, he did a PhD in computer vision from ETH Zurich, and worked at Imperial College London as a postdoc. His main expertise lies in 3D object recognition, SLAM, and applied machine learning. Until recently, he was working on perception for autonomous vehicles at an industrial research lab in Silicon Valley.

Original. Reposted with permission.