An Overview of 3 Popular Courses on Deep Learning

By Vishnu Subramanian.

I have been actively focusing on specialising Deep Learning for the last 2 years. My personal interest towards Deep learning started around 2015 when Google open sourced Tensorflow. Tried quickly couple of examples from the Tensorflow documentation and left with a feeling that Deep learning is difficult, partly because the framework was new and required better hardware and tons of patience.

Fast forward to 2017 I have spent 100’s of hours working on Deep learning projects and the technology has become more and more accessible due to several advancements in software (ease of usage — Keras, PyTorch), hardware(GPU becoming commercially viable for someone like me sitting in India - Not still cheap), availability of data, good books and MOOCs. After completing the 3 most popular MOOCS in deep learning from Fast.ai, deeplearning.ai/Coursera (which is not completely released) and Udacity, I believe a post about what you can expect from these 3 courses will be useful for future Deep learning enthusiasts.

In this post I talk about 5 aspects of each course which will help you decide.

1. About the Guru: Each course is taught by amazing people who come with very different experiences. I believe these experiences has a high influence on the style of teaching.So we will look at the background of our Course Guru’s.
2. About the course: A high level overview of the course.
3. Highlights: Points highlighting where the course makes you feel wow.
4. Limitations: I want to be very choosy of this word, as I know that all of them had put a tremendous and sincere efforts on how to make the content easily accessible. I want this section to be interpreted as, what we missed in the course. Certain limitations could be because of the design of the course.
5. Cost: Cost incurred to take the course.

 

Fast.ai

 
I came across this course when reading an article in KDnuggets. For the first time I heard about Jeremy Howard, searched about him in Wikipedia and was impressed. Checked his MOOC which was taught by Jeremy and Rachel Thomas. Finished watching the first lesson, and was highly impressed with their teaching style.

In a matter of hours you learn how to build a state of art image classifier in less than few $.

About the Guru:

Jeremy comes from a very different background than the usual professors, he is not a PHD from any of the top universities, does not work for any of the top companies like Google, Baidu, Microsoft and many more. He is self taught, kaggle master, entrepreneur and CEO of Fast.ai at present, with the sole aim of Making deep learning uncool again. His uniqueness adds a huge differentiator to this course, as he teaches how deep learning can be used by people from varied backgrounds without access to either huge amounts of data or computation power.

About the course:

The course is divided into 2 parts, each 7 weeks long. The first part of the course teaches how to use deep learning in the fields of computer vision and Natural language processing (NLP). The second part teaches cutting edge research work like generative networks, GAN’s, Sequence to Sequence Models, how to read research papers and lot of practical tips on how to stay ahead in the deep learning space which is evolving at a tremendous speed.

This course is taught in a unique style. The authors of the course believe in an unique approach.

You do not learn playing Cricket(Or any game) after reading about things like velocity, momentum, analytics and then go to field at 18. Instead we go the field, hold a ball or bat and then learn the rest on the fly.

With this unique approach, you start by learning to set up your workstation on the cloud, installing software, and quickly build solutions using Deep learning. Every week Jeremy picks a new problem, and then he teaches techniques to use to improve the performance of the models. He teaches practical things like using pre convoluted features, pseudo labelling, and many tricks which are very useful. By the end of first part you will be able to use deep learning to build useful applications in the domain you are working in.

The 2nd part of the course introduces to cutting edge research by helping you to read, understand and implement various research papers in the space of generative models, image segmentation and sequence2sequence models. You learn to build interesting projects like Style Transfer, Low Resolution Images to High Resolution Images, GAN, Image Segmentations, Language translation and how to apply DL on structured data. The most important part of part-2 is setting up your own workstation.Which proves to be very useful if you are passionate about deep learning and want to build various DL applications.

Highlights:

1. Ability to build state of art systems in Vision and NLP.
2. Understand and use modern architectures that power a lot of Deep learning applications.
3. Loads of practical tips on how to apply DL quickly when you have limited data and computation power.
4. A huge community which supports you at different stages of learning and implementing your solutions.
5. Will be comfortable in using 3 popular DL frameworks namely Keras, TensorFlow, PyTorch.
6. By the end of the course you will be comfortable with reading research papers, build new projects, blog about them and a entire community to support you.

Limitations:

1. As this course follows a top-down approach, you will be relying heavily on the frameworks to abstract away the underlying math. If you are planning for a job or planning for more research in the space, it is helpful to understand the math that powers the DL.
2. Some institutions may give importance to certificates as a proof for course completions, but I guess Jeremy believes that we are mature kids and does not provide any form of certificates. Instead of traditional certificates, Jeremy and Rachel encourages to write blogs, build projects, give talks in conferences. Which I personally believe to be really useful.

Cost:

There is no cost associated with the MOOC. But to practice the projects, you will end up spending on AWS or you may set up you own machine which turns out to be expensive. But having a powerful workstation at house is extremely helpful.

 

Deeplearning.ai

 
I came across this course recently when Andrew Ng tweeted. I have been following him from early 2014, I was learning the math behind Machine learning from one of his courses in Coursera. Coming from an engineering background found his 1st course very interesting and at the same time a bit hard to complete. Fast forward to 2017, Andrew Ng left Baidu where he was working as a chief scientist, launched a new Deep learning specialisation on August 8th. I thought of doing it sometime later this year as I was already occupied with a bunch of other projects. Then I read a blog from a student of Fast.ai Arvind N on how he completed all the 3 parts in 4 days and his views on fast.ai and deeplearning.ai.

I wanted to challenge myself, if I can complete the same in less than 4 days, and yes I finished the 3 courses in 3 days time.

About the Guru:

Andrew Ng was a professor at Stanford university, co-founded Coursera, founded and led the Google Brain deep learning project and was chief scientist at Baidu. The course reflects a lot of his learning from working on wide varieties of problems at scale.

The course helps you in understanding Math required for deep learning and by the end of course 2 you would have built several key components of deep learning algorithms from scratch.

About the course:

The course is divided into 5 parts. At the time of writing the first 3 were released. Andrew Ng takes a bottom-up approach for this course. In his earlier course he picks octave for programming assignments, but he chooses python for this course.

1. In course 1 he carefully covers the math, intuitions required for a lot of concepts in deep learning. He carefully balances the course content, by carefully teaching only the math, that is required to understand the building blocks of deep learning. The assignment sessions are really useful in practising the math in a programmatic way. All the formulas are provided, so you can focus on implementing them, even if you do not have much expertise in math.
2. The course 2 covers a lot of techniques like regularisation, momentum, batch normalization and dropout to improve the performance of your DL models. The best part of this course is you implement all of the techniques using python and numpy.
3. In course 3 he explains a lot of tips and tricks that he learned from years of his experience. At the end of the 3rd course he introduces to DL frameworks. Course 3 ends with an assignment of how to use TensorFlow. The assignment was very clearly designed to be very intuitive.
4. Course 4 will be on CNN. Will update this part once the course is released.
5. Course 5 will be on RNN or Sequence data.Will update this part once the course is released.

Highlights:

1. By the time you complete 3 courses, you will have your fundamentals very strong.
2. First 3 courses take a framework independent approach. Which will prepare you to use any frameworks with ease.
3. Loads of practical tips on how to design evaluation metrics, how to split datasets for training, avoid variance and bias problems.
4. Best part of course 3 is case study, where you get a chance to validate your understandings on how to execute deep learning projects successfully.
5. No need to worry about infra as all the assignments are provided in the form of Jupyter notebooks running on Coursera servers.
6. After an amazing set of 3 courses, waiting for next 2 parts.
7. Each week includes a talk named after “Heroes of Deep learning”. It is a good source to know a lot about the history and inspirations behind deep learning.

Limitations:

1. For some one completely new to Deep learning or Machine learning, it could be challenging to build each component from scratch and understand the math behind it.
2. Since it follows a bottom up approach, even after the end of 3rd course you will find it hard to build solutions using DL in your domain.
3. Since the Infra is provided, you will miss the learning of how to manage your workstation yourself. It is really important to be able to setup your machine on the cloud or at home/office, if you want to use the lot of techniques that you pick up.
4. A lot of boiler plate code is provided to make the assignments easier. But to really pick up a skill it is very important, that you reimplement the entire assignment without using the boiler plate code. This may not be a limitation of the course, but it is really important if you want to make the best use of it.

Cost:
All the course contents are available for free. However I am not sure if the assignments are also available for free. If you are taking the paid version, then you end up spending approximately 3800Rs or 55$ per month.

 

Siraj Raval’s Deep learning Nano Degree Foundation by Udacity

 
I am a great fan of Udacity. They have really amazing courses on wide range of topics. So when I read their announcement on Deep learning course early this year, I was damn excited and enrolled for the first batch.

About the Gurus:

Some parts of the course includes videos from Siraj Raval and the rest from Udacity. I came to know about Siraj Raval from some of his interesting videos on YouTube. He is a multi skilled person who smartly delivers education videos in an interesting format by mixing it with music and videos. Some parts of the course was also taught by Andrew W. Trask and Ian Goodfellow.

About the course:

Unlike the other 2 courses, this course does not take a clear approach of either bottom up or top down. The course was divided into 5 parts, and was given amazon credits of 100$. It covers a lot of Deep Learning techniques like CNN, RNN, GAN, Auto Encoders.

Highlights:

1. In the first few weeks of the course, you learn how to build a Neural network using python and the rest of the course focuses on using TensorFlow.
2. Personalised project reviews are provided for all the projects that are submitted as part of the course.
3. By the end of the course, you get comfortable using TensorFlow as you use Tensorflow for all the projects.
4. Amazing community and forum mentors, who are ready to help you all the time.

Limitations:

1. Practical tips on how to run DL optimally / get state of art performance results was not focused.
2. Some of the projects were too simple for use in real world scenarios. The dataset sizes used for this project were very small.
3. The boiler plate abstracts away a lot of complexity in building DL applications. But if you are focused on learning it for real world usage, it is advisable to implement the projects on your own without using the boilerplate code.

Cost:

The course cost is on the higher side. No free version is available. When I took the course, there was an limited offer so it costed be approximately 25000Rs or 350$.

Recommendations:

If some one is starting deep learning today [self study], I would recommend to finish fast.ai as it has the minimum prerequisites and continue with Coursera Deep Learning Specialisation sequentially or in parallel. Thanks to all the amazing people for making sincere efforts to make this technology accessible.

 
Bio: Vishnu Subramanian is actively looking for opportunities in AI/deep learning. He is a life long learner, and is passionate about deep learning and distributed computing.

Original. Reposted with permission.

Related:

• How I started with learning AI in the last 2 months
• Exclusive: Interview with Jeremy Howard on Deep Learning, Kaggle, Data Science, and more
• First Deep Learning for coders MOOC launched by Jeremy Howard