Know-How to Learn Machine Learning Algorithms Effectively
The takeaway from the story is that machine learning is way beyond a simple fit and predict methods. The author shares their approach to actually learning these algorithms beyond the surface.
By Shareef Shaik, Aspiring Data Scientist
A few days back, a friend of mine came laughing at me, saying “what is taking you so long to learn machine learning? It’s just a few models, I learned them in a week”. Those were his exact words. I simply smiled at him and inquired what he had learned. He told the names of a few machine learning algorithms. I asked him what exactly he learned, then came the obvious reply of using Sklearn’s fit and predict methods and a brief overview of how that algorithm works.
With an evil smile on my face, I asked him what he would do to get the best parameters? How does the model learn the best weights? What do we do when we have a low latency requirement? Though these are not some complex questions, He was sitting there silently watching my face, I had the last laugh.
The takeaway from the story is that machine learning is way beyond a simple fit and predict methods.
Most of us just watch a few videos on youtube and claim we know machine learning, only to realize our mistake pretty quickly. To all the people who are self-learning, just remember that there are hundreds of thousands of people like you and me learning Machine Learning/Data Science. Remember that we will be competing with people who have a Masters/Ph.D. in fields related to Data Science. So to be competitive with them, we need to be really strong with our fundamentals.
With a lot of buzz around Machine learning, new courses are popping out every day, there are more courses out there than the actual jobs at the moment. With this humongous resources there comes the dilemma to pick the right course.
To be honest, most of these courses are Mediocre and don’t cover in-depth. well, there are few good courses out there but each of them is offering a different curriculum. Few covers the math part in-depth few are good at the coding part and so on. So today I won’t be mentioning any course in particular. I am going to share the approach which I followed and suggested to the same friend in the story. I believe it should help you with your data science journey too.
Before we jump into the algorithms part, let me tell you where exactly we use them in a Machine learning project. There are many phases involved in completing a Machine learning project and each phase is equally important.
Modeling is the phase where our Machine learning algorithms come in and it is just one of the important phases in an ML life cycle.
Alright, let's dive into our main topic of Machine learning algorithms.
For every algorithm, we need to be aware of a few important things.
1. What is the intuition behind the algorithm
Initially, I used to learn an algorithm and used to forget after some time and whenever I get back to revise it I used to find it hard to understand which I already learned back then. Over time I realized that we humans can’t really remember things if they are too theoretical and we are more likely to remember it for a long time if learn it through visualizations. That is where geometry helps us visualize the core of an algorithm in the easiest way possible. And we are more tend to understand it deeper if we have some cool real-world daily life examples for these algorithms.
Once you complete learning an algorithm from any course of your choice, open a new tab and start searching for the intuition of that algorithm. Trust me, there are a lot of cool explanations available on the internet. For starters, Quora and Medium are the best places to begin with.
2. How does the algorithm work?
Once you got the Intuition about it, play around and try to observe how the algorithm actually works. One more thing you need to check is how the algorithm works with various forms of data like Text/categorical/numbers.
During this phase, experiment with multiple factors and see how algorithms tend to work, you can refer to the readily available algorithms from Scikit-Learn. Check the parameters of the algorithm and try to play with them and see how they impact the model’s performance.
3. Where it can be used / Where it cannot be used
This is the most important factor that most people often miss out, while they focus more on what and how it works. Why it works or Why it doesn’t work is very important in understanding the algorithm in depth.
Try to experiement and understand how It is working with large datasets, high dimensional data. If it is prone to outliers or imbalanced Dataset.
In interviews the toughest questions are not What and How but Why. Assume Tomorrow you make a personal project after learning algorithms and you used a particular X algorithm to solve it, the interviewer is most likely to respond with why X algorithm why not Y? Let's think you came up with accuracy as the reason, then comes the toughest question why do you think it works better than the rest.
Oh boy! That is why you should be aware of where it works and where it doesn’t work.
For example, You need to come up with a reply like Since our data was high dimensional and non-linear in nature, algorithm X tends to do better with non-linear data than Y due to so and so reasons.
4. Why the interpretability of an Algorithm is important
This is one of the key steps, the reason this is important is that as a job of data scientist you might need to present the model to your clients who might have absolutely no Technical knowledge. During this, you might need to convince them that your model is predicting the right outcome and you need to provide some good reasons why they should adapt to it. You simply can’t throw the accuracy at them. If your algorithm is interpretable then you can show them why the model predicts something. Interpretability means nothing but the feature importance of an algorithm.
For example: If you are working on a Healthcare project where your model predicts the disease of a person to either positive/negative. It's a very sensitive problem and they can’t afford to make mistakes that is when interpretability comes in. If your model shows you the reason(since X feature(some levels in the body) is greater than x value or something) for predicting the person as positive/negative, it will become much easier and makes sense.
5. Why learn the Time/Space Complexity of an Algorithm?
When we work in realtime we might have to deal with huge amounts of data and if there is a requirement for low latency, that is where Time and Space Complexity can help you pick the right algorithms.
If your model occupies more memory, to run it in real-time can get very expensive especially if you are using the cloud infrastructure to run the model. Sometimes a few business problems come with a low latency requirement where are some algorithms that provide good accuracy but fail to satisfy the requirement due to their limitations with time/Space Complexities.
Wikipedia is a great resource to get all this relevant information for each algorithm.
6. Why do we need to understand the math behind an algorithm?
Math again? Yes, yes I get it. We can use algorithms even without understanding the math by simply importing from Scikit learn, but let me tell this is not at all recommended for the long run. Well, no matter if we accept it or not we definitely need math for a better understanding of what really happening under the hood. This might disappoint a few but I got good news we can not avoid Math but we can simplify it and avoid rigorous math.
Simply, most of the Machine Learning algorithms job is to minimize the difference (LOSS) between Actual output and Predicted Output.
algorithm=minimize (Loss) + regularization term
For example, we should minimize log loss for logistic regression and Hinge loss for SVM and etc.
Note: This is not the case for every ML algorithm, this works for few algorithms only.
To minimize this loss, internally these algorithms use Optimization techniques like Gradient Descent and its other flavors which involve a little math. So if you are someone who is not so well with math then just try to understand the loss functions and its Gradient descent for each algorithm, this way you can avoid all other rigorous math and stay intact with the goal. once you are comfortable with that you can dive deep into more math stuff.
7. Why Implement it from Scratch (Optional):
If suppose you are learning a guitar, you would start with the basics first and then slowly you try to replicate the music which is already composed by others and during this phase if you understand how they were able to create that music and tune, then the next step is to create your own music right?
In the same way, We can understand the existing algorithms more clearly by implementing them from scratch replicating the existing one. You’ll get to learn some important points that can help you build better models in the future. Just remember we might not be able to use Scikit learn’s version of an algorithm always in our project due to some issues. In such cases, you’ll have to be ready to optimize or modify the algorithm in a way that satisfies your requirement.
This is easier said than done, it will take a lot of your time to reach this point that is why I marked this as optional. At least try to implement one algorithm to understand how it actually works. If you find it hard to code, break down the algorithm into parts and write the pseudo code first later try to convert that into actual code.
Python Machine Learning by Sebastian Raschka is a really good book that can guide you in developing algorithms from scratch.
Don’t rush in, take your own time and learn it the best way possible instead of learning half and coming back multiple times to refer it every time. Remember the goal is not perfection but a strong foundation, trust me we can never perfect things goal always has to get better each day. If you could understand the gist of the algorithm and aware in and out of how the algorithm works then you’d definitely thank yourself in the long run.
During this process, If you don't understand something or got stuck, leave it aside and move ahead with other topics. Don’t just stick to it and waste your precious time. Come back later with a fresh mind and check it out again, trust me you will be surprised. In case if you are still not able to understand something, search for other resources which explains it in a much easier way.
One last step, apply all your learnings on multiple datasets of different data types and sizes to get real learning, practice is everything. You will get to learn a lot of things by doing than a plain reading so be ready to get your hands dirty.
If you read this far, you definitely have the zeal to learn ML and it is the most important factor to learn any skill. Keep learning and don’t give up.
If you have any suggestions or doubts feel free to let me know in the comments or you can connect with me on my Linkedin.
Happy coding! See you later!!
Original. Reposted with permission.
- How to Explain Key Machine Learning Algorithms at an Interview
- Know What Employers are Expecting for a Data Scientist Role in 2020
- How to Acquire the Most Wanted Data Science Skills