How I Got Better at Machine Learning
Check out this author's collection of tips and tricks that I learned over the years to get better at Machine Learning.
By Jae Duk Seo, Ryerson University
To me, Machine learning is a field of study that requires a good combination of computer science and mathematical knowledge. And although it’s a very interesting field, it’s not a very challenging one. I believe that anyone who has the right amount of motivation, grit, and time can become an expert.
Need some motivation?
Then I should tell you that you can make a LOT of money working in this field. And there are many opportunities in different sectors of industry, so you can try out different sectors
Something amusing that I observed over the years (but of course I could be wrong), is that the fundamentals in machine learning do not change. What I mean by this is the fact that most ‘Models’ we call nowadays are made up of well-known blocks, such as fully connected layers, convolution layers, residual layers, or normalization layers. Of course, not every model is made of these combinations. There are special cases such as Spatial Transformer Networks, and attention mechanisms. However, the general building blocks are more or less the same for the different types of models.
So what does this mean? This means that if you know these fundamentals, it will be much easier and less time-consuming for you to learn the more complicated topics that build on top of these.
So today I wanted to share some tips and tricks that helped me to get “better” at machine learning. But please note that I am still learning machine learning, as there is so much more to discover.
1. Enjoy the Process of Learning, Slow but Steady.
What I mean by this is to devote a lot of time to understand the basics of Machine learning.
To be more specific, learn mathematics.
Vector calculus is needed for understanding backpropagation. Statistics/Probability is critical for understanding information theory. Optimization is essential to know what exactly is happening under the hoods because at the end of the day that is what the model is doing.
I know that researchers who are just starting out don’t really need to know any of this stuff, thanks to packages that perform automatic differentiation. And I personally don’t believe that learning these building blocks will automatically make you an excellent researcher or practitioner in the field
However, I do believe that learning and understanding these concepts will make things a lot easier for you. It did for me. For example, actually implementing a neural network from scratch without getting aid from automatic differentiation, gives you a concrete understanding of how a model learns.
2. Talk with Other Researchers and Get to Know their Work
From time to time, I’ll catch a glimpse of their monitors and see what they’re working on.
And quite often I’ll see something pretty interesting such as segmented results of natural images, or a pile of code that they’re working on. So I’ll ask them questions like “ What is the code about? What is it doing? What kind of problem are you solving?”
Then a conversation sparks. They’ll teach me something about the problem they’re solving, and I’ll learn a new concept or solution. This can be incredibly useful because when the time comes for me to solve similar problems, I won’t have to start from scratch. Also, I get to be introduced to new topics and papers that I didn’t know about.
Although often I won’t fully understand what they are doing, but I still appreciate the fact they took the time to talk with me. After all, there is going to be a time when they would be working on the problems that I was solving and I would be happy to help them out.
3. Find a Usecase for the Theory or Technology you are Interested
Some of the stuff I am working on right now, such as style transfers. I actually started to work on them after seeing a promotional or YouTube video.
Sometimes I’ll just start researching a topic just because I wanted to build something cool. And along the way, I often find a different or better ways to solve the same problem.
Of course, there aren’t any guarantees that those new ways will work better. But it’s the process of learning something new, completely understanding it. To a level where I can incorporate new method that really helps me to get better.
Another reason why this helps me to get better at Machine Learning is that I get to learn the best practices. For example, when a cool project becomes open-sourced I just git clone it from the repository and study how other researchers write their code.
Doing this exposes me to what other researchers are doing and how they are organizing their code. Obviously there are times when it’s extremely hard to understand what the researcher wrote because they are not software engineers. But it still helps to see what they have done and I can take notes on what not to do.
Well, there you have it. These are a few tips and tricks that I used to get a better understanding of machine learning, and I hope they’ll work for you just as well.
At the end of the day, I think anyone with the undying curiosity and grit will be able to research the topic they’re interested in and learn machine learning. Nowadays, there is a lot of excellent online content for deep learning.
While some of these tips will work well for you, might not and that’s perfectly okay. We’re all different types of learners, and not everyone has to fit in the same shoes. But I do hope that at least some of these tips will help you!
Bio: Jae Duk Seo is a first year Master's student at Ryerson University.
Original. Reposted with permission.
- 12 Things I Learned During My First Year as a Machine Learning Engineer
- Inside the Mind of a Neural Network with Interactive Code in Tensorflow
- Understanding Backpropagation as Applied to LSTM