# A Neural Network in 11 lines of Python

A bare bones neural network implementation to describe the inner workings of back-propagation.

This tutorial teaches backpropagation via a very simple toy example, a short python implementation.

Just Give Me The Code:

``` 1.  X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1] ]) 2. y = np.array([[0,1,1,0]]).T 3. syn0 = 2*np.random.random((3,4)) - 1 4. syn1 = 2*np.random.random((4,1)) - 1 5. for j in xrange(60000): 6.     l1 = 1/(1+np.exp(-(np.dot(X,syn0)))) 7.     l2 = 1/(1+np.exp(-(np.dot(l1,syn1)))) 8.     l2_delta = (y - l2)*(l2*(1-l2)) 9.     l1_delta = l2_delta.dot(syn1.T) * (l1 * (1-l1)) 10.     syn1 += l1.T.dot(l2_delta) 11.     syn0 += X.T.dot(l1_delta) ```

Part 1: A Tiny Toy Network

A neural network trained with backpropagation is attempting to use input to predict output. Consider trying to predict the output column given the three input columns. We could solve this problem by simply measuring statistics between the input values and the output values. If we did so, we would see that the leftmost input column is perfectly correlated with the output. Backpropagation, in its simplest form, measures statistics like this to make a model. Let's jump right in and use it to do this.

2. Layer Neural Network:

``` 1. import numpy as np 2 3. # sigmoid function 4. def nonlin(x,deriv=False): 5.    if(deriv==True): 6.    return x*(1-x) 7.    return 1/(1+np.exp(-x)) 8 9. # input dataset 10. X = np.array([ [0,0,1], 11.    [0,1,1], 12.    [1,0,1], 13.    [1,1,1] ]) 14 15. # output dataset 16. y = np.array([[0,0,1,1]]).T 17 18. # seed random numbers to make calculation 19. # deterministic (just a good practice) 20. np.random.seed(1) 21 22. # initialize weights randomly with mean 0 23. syn0 = 2*np.random.random((3,1)) - 1 24 25. for iter in xrange(10000): 26 27.    # forward propagation 28.    l0 = X 29.    l1 = nonlin(np.dot(l0,syn0)) 30 31.    # how much did we miss? 32.    l1_error = y - l133. 34.    # multiply how much we missed by the 35.    # slope of the sigmoid at the values in l1 36.    l1_delta = l1_error * nonlin(l1,True) 37 38.    # update weights 39.    syn0 += np.dot(l0.T,l1_delta) 40. 41. print "Output After Training:" 42. print l1 ```
Output:

```Output After Training:
[[ 0.00966449]
[ 0.00786506]
[ 0.99358898]
[ 0.99211957]]
```  Get the FREE ebook 'The Great Big Natural Language Processing Primer' and the leading newsletter on AI, Data Science, and Machine Learning, straight to your inbox.  