Connecting the dots for a Deep Learning App

We show how to build a Deep Learning app which does sentiment analysis on movie reviews. Try it yourself!

Janardhan Shetty.
Our day to day activities is filled with Emotions and Sentiments. Ever wondered how we can identify these sentiments through computers? Oops, computers who have no brains :)?

Try this Deep Learning App yourself (refresh a couple of times initially if there’s Application Error):

Dot 0: Deep Learning in Sentiment Analysis

Sentiment analysis is a powerful application which extends its arms to the following fields in the modern day world.
1. Movie Reviews — More than 60% of the people check online reviews of a movie and then decide to watch it.
2. Restaurant Reviews — Have you ever visited a restaurant without reading a review?
3. Product Reviews — Many B2B or B2C products are sold based on the reviews they generate.
4. Feature Reviews — Any product’s success is directly proportional to its features’ reviews.
5. Political Reviews — Political parties and candidates are chosen based on the fact of how they fare in their reviews.
6. Customer Support Reviews — Impressive products fail due to bad support reviews. Alternately simple products take off with good reviews.
7. Blog Reviews — Waiting for your comments and reviews on this blog to learn the sentiment!!!

What is Sentiment Analysis?
According to Wikipedia: Sentiment analysis (sometimes known as opinion mining or emotion AI) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information.
On this note, let’s develop a Movie Review Sentiment Application using the methods of Deep Learning.

What is Deep Learning?
Deep learning is an advanced field of machine learning and a step towards the bigger subject of Artificial Intelligence.
Currently, it’s creating waves in the field of Image processing, Natural Language Processing, Speech Processing, Video Processing etc.
Here we will talk about how to connect the various dots to complete a Deep Learning App.
Before we head out to connect the dots lets equip ourselves with few definitions and concepts which will be useful.

The dataset which will be used for any prediction task should be first analyzed for the following key points before even diving into solving the same.

1. How good are the labels?
2. Can we do spot checking of few data points?
3. How big is the dataset?

Once we pass this basic hurdle, the big process in any Data Scientist’s life comes into the picture.

Dot 1: Data Preparation

Preparing and cleaning data is currently the bulk of the Data Science work. Believe me, it is 100% true.
Movie Review Sentiment App uses a customized data-set of numerous movie reviews from various different sources like IMDB, UCI, Cornell dataset etc
The process of making a gold data-set(Prepared data-set for prediction) is a step by step process of Data Collection, Data Cleaning, Data Normalization after which the data can be termed as a gold data-set for the prediction process.

Once we have the Gold Dataset which is nothing but prepared data set for prediction, we are ready for modeling.

Dot 2: Baseline Model

The first architecture in any Deep Learning Analysis is to start always with a single hidden layer architecture thanks to Prof. Jeremy Howard and Prof. Rachel Thomas who insists to start simple for training any Deep Models. This simple yet powerful architecture acts as our baseline !!!
The following are the steps employed in training our Sentiment Analysis (Natural Language Processing) pipeline:

1. Tokenizer: Each review is broken into a set of tokens called as words.This can be done using keras text tokenizer.

2. Embeddings: There are various ways to encode these words into vectors (vectorization) and here, in this case, we will use default keras embeddings as the starting point. Although final model uses efficient pre-trained word2Vec embeddings which capture semantic information of the words and their relationship with other words.

3. Review Encoding: These word embeddings capture the semantic relationship between words. We then need to come up with an encoding of whole movie review so that this can be passed as training data. The simplest ploy is to sum the individual word encodings to form review encoding.

4. Predictions: These review encoding is then passed through a single hidden fully connected neural network layer of 300 neurons which looks like below:
Single Hidden Layer with BatchNormalization

Complete Baseline Model looks like this:

Dot 3: Experimentation of Different Model Architectures

Once we have the baseline we can start improving our model objective such as accuracy. There was a lot of experiments conducted in order to improve the accuracy of the Sentiment App: Below table provides the summary of models tried and tested.