Deep Learning and Neural Networks Primer: Basic Concepts for Beginners

This is a collection of introductory posts which present a basic overview of neural networks and deep learning. Start by learning some key terminology and gaining an understanding through some curated resources. Then look at summarized important research in the field before looking at a pair of concise case studies.

Deep Learning Primer

Interested in neural networks and deep learning? Have a vague understanding of what it is and what it entails, but don't know how to proceed learning the foundational concepts? This small collection of explanatory posts can get a novice to a place where deeper investigation can be undertaken, after building a base of introductory terminology and concepts.

Deep Learning Key Terms, Explained

Deep learning is a relatively new term, although it has existed prior to the dramatic uptick in online searches of late. Enjoying a surge in research and industry, due mainly to its incredible successes in a number of different areas, deep learning is the process of applying deep neural network technologies - that is, neural network architectures with multiple hidden layers - to solve problems. Deep learning is a process, like data mining, which employs deep neural network architectures, which are particular types of machine learning algorithms.

So, with that, let's have a look at some deep learning related terminology, with a focus on concise, no-nonsense definitions.

7 Steps to Understanding Deep Learning

In essence, deep learning is the implementation of neural networks with more than a single hidden layer of neurons. This is, however, a very simplistic view of deep learning, and not one that is unanimously agreed upon. These "deep" architectures also vary quite considerably, with different implementations being optimized for different tasks or goals. The vast research being produced at such a constant rate is revealing new and innovative deep learning models at an ever-increasing pace.

This collection of reading materials and tutorials aims to provide a path for a deep neural networks newcomer to gain some understanding of this vast and complex topic. Though I do not assume any real understanding of neural networks or deep learning, I will assume your familiarity with general machine learning theory and practice to some degree.

Deep Learning in Neural Networks: An Overview

What a wonderful treasure trove this paper is! Schmidhuber provides all the background you need to gain an overview of deep learning (as of 2014) and how we got there through the preceding decades.

I shall now attempt to condense a 35-page summary of 900 papers into a single blog post! Needless to say, there’s a lot more detail in the full paper and references than I can cover here. We’ll look at the following topics: Credit assignment paths and the question of how deep is deep?; Key themes of Deep Learning; Highlights in the development of Supervised and Unsupervised Learning methods, Reinforcement Learning; and a short look at where things might be heading.

Note that this post is written by Adrian Colyer.

Artificial Neural Networks (ANN) Introduction, Part 1

Image recognition is important for many of the advanced technologies we use today. It is used in visual surveillance, guiding autonomous vehicles and even identifying ailments from X-ray images. Most modern smartphones also come with image recognition apps that convert handwriting into typed words.

In this chapter we will look at how we can train an ANN algorithm to recognize images of handwritten digits. We will be using the images from the famous MNIST (Mixed National Institute of Standards and Technology) database.

Note that this post is written by Kenneth Soo.

Artificial Neural Networks (ANN) Introduction, Part 2

We’ve learned how Artificial Neural Networks (ANN) can be used to recognize handwritten digits in a previous post. In the current post, we discuss additional techniques to improve the accuracy of neural networks. Neural networks have been used successfully to solve problems such as image/audio recognition and language processing.

Matching the performance of a human brain is a difficult feat, but techniques have been developed to improve the performance of neural network algorithms, 3 of which are discussed in this post: Distortion, mini-batch gradient descent, dropout.

Note that this post is written by Kenneth Soo.