Are New Technologies Killing Their Ancestors?
Are automatic feature learning models (e.g. CNN) killing their previous manually engineered models? This is an important question that is to be answered in this article.
For every moment in our life, there are new technologies that may re-do the previous tasks differently or add something new. For every new technology, is its previous one destroyed and will not be used later? Are automatic feature learning models (e.g. CNN) killing their previous manually engineered models? This is an important question that is to be answered in this article.
Every data scientist would have used a calculator for doing mathematical calculations. The calculators have a simple interface with just required buttons for doing the math. After the invention and evolvement of the mobile phones, smartphones came out that have different applications for doing operations previously done in the calculator. Here is the question. Is by the appearance of a new technology (smartphones), its previous technology (calculator) got destroyed and will not ever be used later? The answer is definitely NO. Some people say that the newest is always the best but this does not always hold. This is because not only benefits are added by new technologies, but also they might add some disadvantages. Here is a discussion to show what the option to use for doing math operations: calculator or smartphone.
Calculators were just dedicated to mathematical operations but smartphones are not. Smartphones have many features not existing in the calculator. Is the availability of many features rather than limited ones a disadvantage? In some cases, the fewer features in the tool the better its performance and the more features the more overhead. It is simple to do an operation using a calculator. Just turn on it and press buttons required to do the operation. But there is overhead when doing the same operation in smartphone due to its many features. Here are some cases that may happen when using a smartphone as a calculator.
After opening the application used for doing math calculations, the phone might ring for an incoming call which breaks whatever you were doing. The phone may beep for an incoming SMS. It might be connected to the Internet and thus may beep for an e-mail. This might take you off from doing the operations. After taking a long time using the phone, the battery may get empty and the phone may turn off unless it is connected to the charger which will limit the movement flexibility. As a result, one using a smartphone should care of all of such effects in order to do the math operations nicely. Compared to calculators, there is no SIM card and thus no incoming call or SMS. Basic calculators are not connected to the Internet and thus no incoming e-mails. The battery of calculators last longer than batteries of smartphones and thus the probability of interrupting the operation is smaller when using a calculator. As a result, using calculators of limited features compared to smartphones have the advantage of being simple and focused on the task even if it is an old technology. As a conclusion, the newest is not always the best. According to your needs, the old technology may be better or worse than a newer technology. The same holds from data science perspective.
There are different types of learning algorithms and features to be used for different tasks such as classification and regression. Some of them may extend to 1950 while others are recent. But we can’t say that the old models are always worse than recent models. We can’t absolutely conclude that deep learning (DL) models such as CNN are better than previous models. This depends on your needs.
I found many researchers tending to use DL blindly just because it is the state-of-art method. Some problems are simple and using DL may add more complexity. For example, using DL with just 100 images divided across 10 classes is not a good option. This is because the number of samples is small and not requires DL. Shallow learning is just sufficient in this case. If a classifier is to be created to discriminate the four types of fruits (taken from fruits-360 dataset) shown below, DL is not mandatory and previous hand-crafted/engineered features are sufficient. By investigating the problem, one type of features that might be helpful is the color. Each fruit has a different color. Thus the histogram of the hue channel in the HSV color space might be sufficient.
Let’s show the complexity added by using CNN deep learning model in such problem. At first, there should a large amount of data to train such model. There are different parameters to be specified and each parameter requires a long time of experiments. These include which types of layers to be used, number of each layer, activation function, learning rate, and others. In comparison, using hue channel histogram is sufficient for getting a very high accuracy.
There are different ways to solve the data science problem and you should select the most appropriate one. Be wise in your selection to get the fittest. If a 350 milliliters water bottle is sufficient for you, why buy a 1-liter bottle?
It is like getting to the top of a wall using a ladder. If you reached the top of the wall after climbing 5 stairs, you do not need to go up another stair in the ladder. Similarly, if you can get the best results using hand-engineered features, you do not have to use automatic feature learning.
As a conclusion, stop thinking of new technologies as the wow solution to every previous problem. Even if it solved previous problems, it may introduce others or add more complexity. Finally, stop thinking of DL as the solution to every problem.
Bio: Ahmed Gad received his B.Sc. degree with excellent with honors in information technology from the Faculty of Computers and Information (FCI), Menoufia University, Egypt, in July 2015. For being ranked first in his faculty, he was recommended to work as a teaching assistant in one of the Egyptian institutes in 2015 and then in 2016 to work as a teaching assistant and a researcher in his faculty. His current research interests include deep learning, machine learning, artificial intelligence, digital signal processing, and computer vision.
Original. Reposted with permission.
- Is Deep Learning Overhyped?
- Is Learning Rate Useful in Artificial Neural Networks?
- 7 Steps to Understanding Deep Learning