7 Steps to Mastering Deep Learning with Keras
Are you interested in learning how to use Keras? Do you already have an understanding of how neural networks work? Check out this lean, fat-free 7 step plan for going from Keras newbie to master of its basics as quickly as is possible.
There is no shortage of neural network frameworks, libraries, and APIs available to anyone interested in getting started with deep learning. So... Why Keras?
Keras is a high-level neural network API, helping lead the way to the commoditization of deep learning and artificial intelligence. It runs on top of a number of lower-level libraries, used as backends, including TensorFlow, Theano, CNTK, and PlaidML. Keras code is portable, meaning that you can implement a neural network in Keras using Theano as a backened and then specify the backend to subsequently run on TensorFlow, and no further changes would be required to your code. Data Scientist & Machine Learning Expert Charles Martin sums up with this quote (in a response to the post):
We have had access to these algorithms for over 10 years. Not readily accessible, but accessible enough to a good hacker. To me, the most important advances have come with Google Keras, which has commoditized very powerful, modern, AI algos that previous were not only inaccessible bit [sic] thought to be unusable as well.
Keras is an easy to use neural network library that promotes a simple and intuitive syntax. It is also geared toward neural network technology consumers as opposed to producers, at least to some degree. The line between these 2 groups is not exactly well-defined; what, exactly, constitutes the design and implementation of a machine learning algorithm versus modifying an already-prepared one? This is chicken versus egg stuff, and not really an appropriate discussion topic for here. However, I will make the assertion that, perhaps more than any other established, mainstream neural network library, Keras is ideally suited to the practice of data science. In fact, I would say that Keras is an essential tool in the toolbox of any data scientist working with neural networks.
This tutorial is designed to get you up to speed with Keras as quickly as possible, allowing you to hit the ground running, not a particularly difficult task if you already have familiarity with neural networks. To that end, an understanding of neural networks is preferable -- though a very brief intro will be presented in the first step -- as is some experience with other machine learning frameworks. Beyond that, no experience specifically with Keras is required.
Step 1: Neural Networks Basics
This post collects a number of neural network "getting started" resources.
Step 2: Keras Basics
First, what exactly is Keras? Why not find out directly from the project's website? Read the entirety of this main page (it will only take a couple of minutes), paying particular attention to "30 Seconds to Keras," which should be enough to give you an idea of how simple Keras is to use.
The release notes for the current major release version of Keras (at the time of writing) can be found here. Take the few minutes to read them:
Karlijn Willems of DataCamp has created a handy Keras cheat sheet which I recommend you download and have handy. Even though some of it may not make sense yet, it soon will, and the reference will undoubtedly prove useful. I have a printed copy on my desk.
- Keras Cheat Sheet, by Karlijn Willems
Lastly, and most importantly, familiarize yourself with the Keras documentation, which is top notch and explains all the things. Bookmark it.
Step 3: An Overview of Keras
Valerio Maggio gave a great tutorial presentation at PyData London 2017, titled "Ten Steps to Keras." To get a fleshed out understanding of what Keras is, how it compares to other libraries, and how you can get things done with it, take the 90 minutes to watch the video. It is definitely worth it to gain a high level understanding of Keras, top to bottom.
Then look at the following few pages of documentation to get an understanding of how Keras approaches model implementations.
- About Keras models
- Getting started with the Keras Sequential model
- Getting started with the Keras Functional API
If you are still having questions as to why you would choose Keras over TensorFlow (you didn't watch the video, did you?), read this:
- TensorFlow or Keras? Which one should I learn?, by Aakash Nain
Make sure you have Keras installed before going any futher.
Finally, for information on changing Keras backends, see this documentation page:
Step 4: Baby Steps with Keras
Let's do one of the first things that everyone does when they start using a new deep learning library: implement a simple logistic regression model.
If you watched Valerio's video in the previous step then you may already have gotten what you need for this. If not, refer to his notebook from the talk. Ideally you read the whole thing for a great comparison of implementation code for TensorFlow, Theano, and Keras. If you are interested solely in the Keras code, however, skip down about halfway.
- Multi-layer Fully Connected Networks, by Valerio Maggio
The beauty of the Keras code, shown below (and taken directly from Maggio's notebook), to accomplish the task should be readily visible. Note that data pre-processing has already occurred at this point.
Implementing logistic regression, as above, is one thing, but now let's try out something more worthy of being called a neural network, complete with a hidden layer.
First, ensure that you have looked over the following Keras documentation pages, describing both the Sequenial model (already referenced above) and the Layers.
Now, follow along with this basic implementation of a neural network which learns something about tic-tac-toe game outcomes.
- Keras Tutorial: Recognizing Tic-Tac-Toe Winners with Neural Networks, by Matthew Mayo
Having completed these steps, you should be ready for implementing some more complex architectures.
Step 5: Implementing a Convolutional Neural Network
To implement a convolutional neural network (CNN) in Keras, start by reading the documentation on its convolutional layers:
After this, look at both of the following tutorials on CNNs in Keras. The first takes a slower approach, and covers much of what you should already know by now, while the second takes into account some additional topics such as avoiding overfitting. This isn't necessarily an either/or approach; you may find valuable nuggets in both write-ups:
- Keras Tutorial: The Ultimate Beginner’s Guide to Deep Learning in Python, by Elite Data Science
- Introduction to Convolutional Neural Networks, by Mike Irvine
To learn more about convolution neural networks in general, try Brandon Rohrer's video:
Read the accompanying blog post.
Step 6: Implementing a Recurrent Neural Network
To implement a recurrent neural network (RNN) in Keras, start by reading the documentation on its recurrent layers:
After this, try out this tutorial by Chris Albon for implementing a Long Short-Term Memory (LSTM) network -- a dominant type of RNN -- for text classification:
To learn more about recurrent neural networks in general, try Brandon Rohrer's video:
Read the accompanying slides.
Step 7: What Next?
At this point you should have a solid understanding of Keras, including why you would use it, the advantages it has over other libraries in certain situations, and using it to implement a variety of network architectures. You're an expert now, right?
Well, probably not. But hopefully you have mastered the basics. When you want to move beyond the basics, the next best stop is likely Keras' official tutorials:
After this, check out the Keras examples directory, which includes vision models examples, text & sequences examples, generative models examples, and more.
Also, how about challenging yourself to fine-tune some of the above models you implemented in the previous steps? Change the optimizer, add another layer, play with activation functions. Or use some of the Keras metrics to go back and judge the performance of your models. The Keras documentation is a great place to start:
Best of luck using Keras.