Is Deep Learning Overhyped?

Editor: see also results of this poll: Deep Learning is not Enough

Interest in deep learning continues to grow. Google Trends shows a steady increase in the search term "deep learning" over the past few years, with an even more noticeable uptick since late 2014. Showing even more impressive recent gains are the search terms "deep neural network" and "convolutional neural network" (along with other relatively new deep architectures, quite obviously).


Google Trends for the search term "deep learning."

There is much excitement surrounding deep learning, and with such excitement comes both the zealots and the detractors. Some claim that deep learning is the edge of the chasm of "true AI," able to solve all of humanity's problems. Others claim that it is nothing more than hype, a fad which will disappear or be severely tempered in the near future. Is deep learning either of these extremes? Is it somewhere in between?

Recently on Quora, a question on this subject was posed directly to Yoshua Bengio, one of the fathers of the rejuvenated deep learning movement, and whose accomplishments are far too numerous to mention here. The question was, simply, "Yoshua Bengio: Is the current hype on Deep Learning justified?" Though clearly a proponent of deep learning, and one of the unquestionable drivers of research in the field over the past decade (he was deep learning before there was deep learning), his answer is both concise and balanced, at least in my view.

In my opinion, the most important point of his argument is as follows:

If it's hype, it's exaggeration. The exaggeration exists, I have seen it. It is there when someone presents this body of work as something that puts us much closer to human-level intelligence than we really are, often relying on the mental images many people have built of AI based on movies and science-fiction.

The entertainment industry has shaped the public's view of artificial intelligence for decades, which is a shame given the ubiquitous access to actual information that so many have (literally) at their fingertips these days. Bengio's above comment also brings to the forefront the idea that listening to those who are the loudest is not always the best idea. The ongoing US Presidential race is further evidence of this.

After waxing philosophical, Bengio clarifies that, while it may not currently satisfy the public's requirements for human-equivalent AI, deep learning has already achieved great success in particular domains, and that further research may actually out-perform humans in limited domains moving forward. He states that the economic impact and benefit of this should be significant, even if it does not culminate in "true AI."

A number of other questions on deep learning and its hype can be found on Quora, a few of which are listed below. Most of the answers coming from those "in the know" are generally of similar opinion as Yoshua Bengio.

• Does deep learning really work? Is it just hyped by researchers which is fueled by the GPU manufacturers?
• Is deep learning overhyped?
• Has Deep Learning passed the hype peak? If so, what would be next hot topic? Write AnswerRe-Ask

Of course, it is important to remain tempered in our expectations of deep learning. As the world seemingly scrambles for "The Master Algorithm," we must keep in mind that deep learning is not a machine learning panacea. While deep neural networks have their place, they won't solve all of humanity's woes. At least, not yet.


Breaking news: Deep learning delivers, world's problems solved.

Many also wonder, will deep learning render useless all other forms of machine learning? In an answer to the Quora question "Will deep learning make other machine learning algorithms obselete," Brian Quanz, a PhD in machine learning, gives the most straightforward answer, in my opinion:

No, different problems will always have different methods that work best. It is about finding the best method for your data. There is no method that is universally best for all problems.

Brian goes on to list other considerations that go into choosing an algorithm, including desired complexity, intepretability, and resource constraints, among others.

While deep learning is making waves, and deservedly so, we must keep in mind that it is but another effective tool to be used in appropriate situations. Even so, people will have opinions running the gamut from it being overhyped (by GPU manufacturers or others), to being the solution to every problem they will ever experience, to somewhere more moderate in between.

Just like with any scenario in life, approaching deep learning with a level head would be in the best interest of any researcher, practitioner, or member of the public. And remember that no matter how good the meal was, you always pay for lunch.

Related:

• Top 5 Deep Learning Resources, January
• (Deep Learning’s Deep Flaws)’s Deep Flaws
• 7 Steps to Understanding Deep Learning