Which Machine Learning Algorithm be used in year 2118?
So what were the answers popping in your head ? Random forest, SVM, K means, Knn or even Deep Learning? No, for the answer, we turn to Lindy Effect.
So what were the answers popping in your head? Random forest, SVM, K-means, K-nn or even Deep Learning and its variants?
Now some of you might laugh and say how on earth can you predict so far ahead, predicting things 100 yrs into future is crazy.
What makes you say Regression Will continue to be used in 2118?
Well the answer is Lindy effect. Yes, the heuristic I am using to predict this is Lindy Effect.
Alright next logical question would be what is Lindy Effect?
Lindy Effect
Wikipedia defines the Lindy effect as
The Lindy effect is a concept that the future life expectancy of some non-perishable things like a technology or an idea is proportional to their current age, so that every additional period of survival implies a longer remaining life expectancy.
One of my favorite author Nassim Taleb in his famous book Antifragile: Things that gain from disorder defines Lindy Effect as follows.
If a book has been in print for forty years, I can expect it to be in print for another forty years. But, and that is the main difference, if it survives another decade, then it will be expected to be in print another fifty years. This, simply, as a rule, tells you why things that have been around for a long time are not “aging” like persons, but “aging” in reverse. Every year that passes without extinction doubles the additional life expectancy. This is an indicator of some robustness. The robustness of an item is proportional to its life!
His article on Lindy Effect ‘An Expert called Lindy’ is a highly recommended read.
So Why will Regression survive that long?
Well, because it has survived this long. Regression (method of least squares) as a concept was first invented in the 1800’s by Carl Friedrich Gauss and Adrien-Marie Legendre. They used it to determine the orbital paths of planets and other bodies around the sun.
The word ‘Regression’ was coined by Francis Galton to describe the observation that the taller fathers tend to have relatively shorter sons while shorter fathers tend to have relatively taller sons!!
Okay so quite clearly Regression has been around for more than 200 years already !! so going by the Lindy Effect heuristic it will last another 200 years. So in fact I might be little conservative in saying that Regression will continue to be in use in year 2118.
What is the Secret behind the longevity of Regression?
The case in point would be the below report from a 2016 Kdnuggets survey
In fact in another survey conducted by kdnuggets in 2011, Regression came a close second. So going by the Lindy effect it has become ‘more immortal’ in 5 years by topping the charts !!
(Update : At the time of writing this article I was not aware that there was a 2017 survey as well. In 2017 survey too Regression tops the chart)
Regression is still the widely used ML algorithm. People are using Regression or continue to use Regression because
It is simple
Highly interpret-able (Even Dilbert’s boss can understand it :P )
IT WORKS
The ‘IT WORKS’ Part
People across various domains continue to use Regression because it has worked for them . There is a clear ROI that people have gained by using Regression. For example, in marketing, the driving force behind Market Mix Modeling is regression. It is still a popular technique and many FMCG companies believe the outputs from MMM. The same holds true for other domains too. If Regression was not useful in delivering the results, it would have gone the dodo way. It is still used by the Industry and Academia alike because ‘IT WORKS’.
What about Neural Nets and Its Variants? Will they be used in 2118?
Well so far the Lindy Effect has not been Kind to Neural Nets or lets call it AI. It has already faced AI Winter. The longevity of Neural Nets and its variants have been hampered by ‘AI Winter’ in the 20th century. Such disruption is not a good sign for the longevity of technology or in this case algorithms.
But on the brighter side AI related advancements has grown from strength to strength in last decade. And I as an eternal student, continue to be fascinated about the latest AI breakthroughs. So a safe bet could be that we could see Neural Nets and its variants to survive another 10–20 years, with the hope that ‘Singularity’ fear expressed by Elon Musk does not cause another AI winter.
What can Mitigate Lindy Effect of a Machine Learning Algorithm?
Machine Learning Overkill : Yes, Lindy effect will get mitigated because of wrong application of machine learning algorithms and overkill of it. I have come across situations where people have used a Machine Learning algorithm where a simple common sense baseline approach would have worked. Mr. Rama Ramkrishnan captures this essence excellently in his article .
The recent fad of Data Science being the sexiest job is not helping the cause either. The machine learning algorithms have become like a hammer at the hands of data scientists. Everything looks like a nail to be hit upon. In due process the wrong application or overkill of machine learning will cause disenchantment among people when it does not deliver value. It will be a self inflicted ‘AI Winter’.
But for time being Regression will have the last laugh now and probably even in the year 2118.
You can reach out to me on
Linkedin
Twitter
My Blog: Analytics Loka
Original. Reposted with permission.
Related