Will Deep Learning take over Machine Learning, make other algorithms obsolete?

Deep learning grows rapidly and surprises us with amazing empirical results. There is a discussion on Quora about whether deep learning will make other machine learning algorithms obsolete. Specifically, will related algorithms such as back propagation, HMM become obsolete like perceptrons?

Huh, difficult to answer. An interesting answer comes from Jack Rae, who says

Empirical results over the past couple of years have shown that deep learning provides the best predictive power when the dataset is large enough. Is that true? Well I don’t know of a single example of where it has been beaten in predictive power for a dataset of over 100 million rows in the past year

He thinks that deep learning will push other learning algorithms to near extinction, is because of the unsurpassed predictive power of deep learning especially on medium-to-large datasets. Other algorithms will become obsolete when people begin to consider deep learning as the first solution to some problems, such as pattern recognition.

On the other hand, most people still believe that deep learning will not replace all other models and algorithms. Opinion from Jacob Steinhart received the most upvotes. He writes

1. For many applications, far simpler algorithms like logistic regression or support vector machine will work just fine, and using a deep belief network will only complicate things.

2. While deep belief networks are one of the best domain-agnostic algorithms, if one has domain knowledge then many other algorithms (such as HMMs for speech recognition, wavelets for images, etc.) can outperform them. There is some work being done to incorporate such domain knowledge into neural network models, but it is certainly not yet enough to fully replace all other models and algorithms.

Above is a timeline of machine learning created by Eren Golge.

Deep learning is going to become mainstream just like SVM, which improved rapidly in the early 2000s. However, the complexity of deep learning and its requirement of large amount of data are still need to be solved before Deep Learning becomes the first choice for machine learning algorithms.

Ran Bi is a master student in Data Science program at New York University. She has done several projects in machine learning, deep learning and also big data analytics during her study at NYU.

Related:

• Deep Learning – important resources for learning and understanding
• Does Deep Learning Have Deep Flaws?
• CuDNN – A new library for Deep Learning