A Beginner’s Guide to Neural Networks with Python and SciKit Learn 0.18!

Data Preprocessing

 
The neural network may have difficulty converging before the maximum number of iterations allowed if the data is not normalized. Multi-layer Perceptron is sensitive to feature scaling, so it is highly recommended to scale your data. Note that you must apply the same scaling to the test set for meaningful results. There are a lot of different methods for normalization of data, we will use the built-in StandardScaler for standardization.

StandardScaler(copy=True, with_mean=True, with_std=True)

Training the model

 
Now it is time to train our model. SciKit Learn makes this incredibly easy, by using estimator objects. In this case we will import our estimator (the Multi-Layer Perceptron Classifier model) from the neural_network library of SciKit-Learn!

Next we create an instance of the model, there are a lot of parameters you can choose to define and customize here, we will only define the hidden_layer_sizes. For this parameter you pass in a tuple consisting of the number of neurons you want at each layer, where the nth entry in the tuple represents the number of neurons in the nth layer of the MLP model. There are many ways to choose these numbers, but for simplicity we will choose 3 layers with the same number of neurons as there are features in our data set:

Now that the model has been made we can fit the training data to our model, remember that this data has already been processed and scaled:

MLPClassifier(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,
       beta_2=0.999, early_stopping=False, epsilon=1e-08,
       hidden_layer_sizes=(30, 30, 30), learning_rate='constant',
       learning_rate_init=0.001, max_iter=200, momentum=0.9,
       nesterovs_momentum=True, power_t=0.5, random_state=None,
       shuffle=True, solver='adam', tol=0.0001, validation_fraction=0.1,
       verbose=False, warm_start=False)

You can see the output that shows the default values of the other parameters in the model. I encourage you to play around with them and discover what effects they have on your model!

Predictions and Evaluation

 
Now that we have a model it is time to use it to get predictions! We can do this simply with the predict() method off of our fitted model:

Now we can use SciKit-Learn's built in metrics such as a classification report and confusion matrix to evaluate how well our model performed:

[[50  3]
 [ 0 90]]

             precision    recall  f1-score   support

          0       1.00      0.94      0.97        53
          1       0.97      1.00      0.98        90

avg / total       0.98      0.98      0.98       143

Looks like we only misclassified 3 tumors, leaving us with a 98% accuracy rate (as well as 98% precision and recall). This is pretty good considering how few lines of code we had to write! The downside however to using a Multi-Layer Preceptron model is how difficult it is to interpret the model itself. The weights and biases won't be easily interpretable in relation to which features are important to the model itself.

However, if you do want to extract the MLP weights and biases after training your model, you use its public attributes coefs_ and intercepts_.

coefs_ is a list of weight matrices, where weight matrix at index i represents the weights between layer i and layer i+1.

intercepts_ is a list of bias vectors, where the vector at index i represents the bias values added to layer i+1.

4

30

30

Conclusion

 
Hopefully you've enjoyed this brief discussion on Neural Networks! Try playing around with the number of hidden layers and neurons and see how they effect the results!

Want to learn more? You can check out my Python for Data Science and Machine Learning on Udemy! Get it for 90% off at this link:
www.udemy.com/python-for-data-science-and-machine-learning-bootcamp/?couponCode=KDNUGGETSPY

If you are looking for corporate in-person training, feel free to contact me at: training AT pieriandata.com

Bio: Jose Portilla is a Data Science consultant and trainer who currently teaches online courses on Udemy. He also conducts training as the Head of Data Science for Pierian Data Inc.

Related:

• Deep Learning Key Terms, Explained
• A Beginner’s Guide to Neural Networks with R!
• Why Do Deep Learning Networks Scale?