Inside the Mind of a Neural Network with Interactive Code in Tensorflow
Understand the inner workings of neural network models as this post covers three related topics: histogram of weights, visualizing the activation of neurons, and interior / integral gradients.
Pages: 1 2
Interior Gradients / Positive and Negative Attributes
alpha value of 1.0
Now lets use interior gradient, with the alpha values of ….
0.01, 0.01, 0.03, 0.04, 0.1, 0.5, 0.6, 0.7, 0.8 and 1.0 (same as original input), to visualize the the inner workings of the network.
All of the images above represents the gradient respect to the input with different alpha values. We can observe as alpha value increases the gradients becomes closer to 0 (black) due to the saturation within the network. We can clearly see the difference of how gradients change when we view all of the images in a gif format.
However, I was able to make an interesting observation, as alpha value becomes smaller my network wasn’t able to make the correct predictions. As seen below when the alpha value is small (0.01) the network predict mostly 3 or 4. (Cat/Deer)
Red Box → Models Prediction
But as alpha value increased, eventually ending up at 1.0, we can observe that the accuracy of the model increases. Finally, lets take a look at positive and negative attributes from the gradient at each alpha values.
Blue Pixel → Positive Attributes Overlayed with Original Image
Red Pixel → Negative Attributes Overlayed with the Original Image
Again, it is easier to view the change if we create a gif out of the images.
Integral Gradient / Positive and Negative Attributes
Left Image → Integral Gradient with Step Size 3000 for riemann approximation
Right Image → Blue Color for Positive Attributes and Red Color for Negative Attributes
Finally when we use Integral Gradient to visualize the gradients we can observe something like above. One very interesting fact I was able to find was how the network predicted horse (7), dog (5) and deer (4).
Lets first take a look at the original images, as seen above the left, middle images are horses and right image is a sheep.
When we visualize the Integral Gradient we can see something interesting. As seen in the middle image the network thought the head portion of a horse was most important. However, when only taking the head, the network thought it was an image of a dog. (And it kinda looks like a dog). Similar story for the right image, when the network only taken into account of only the head of a sheep, it predicts as seeing a dog.
However, as seen on the most left image, when the network takes in the overall shape of a horse (including some portion of a human) it correctly identifies the image as a horse. AMAZING!
Interactive Code
For Google Colab, you would need a google account to view the codes, also you can’t run read only scripts in Google Colab so make a copy on your play ground. Finally, I will never ask for permission to access your files on Google Drive, just FYI. Happy Coding! Also for transparency I uploaded all of the training logs on my github.
To access the code for this post please click here, for training logs click here.
Final Words
As this area is still growing and being developed there will be new methods and findings. (I hope I can contribute to those findings as well.). Finally, if you want to access the code from the original authors of the paper “Axiomatic Attribution for Deep Networks” please click here.
If any errors are found, please email me at jae.duk.seo@gmail.com, if you wish to see the list of all of my writing please view my website here.
Meanwhile follow me on my twitter here, and visit my website, or my Youtube channel for more content. I also implemented Wide Residual Networks, please click here to view the blog post.
Reference
 http://Axiom/definition/theorem/proof. (2018). YouTube. Retrieved 14 June 2018, from https://www.youtube.com/watch?v=OeC5WuZbNMI
 axiom — Dictionary Definition. (2018). Vocabulary.com. Retrieved 14 June 2018, from https://www.vocabulary.com/dictionary/axiom
 Axiomatic Attribution for Deep Networks — — Mukund Sundararajan, Ankur Taly, Qiqi Yan. (2017). Vimeo. Retrieved 14 June 2018, from https://vimeo.com/238242575
 STL10 dataset. (2018). Cs.stanford.edu. Retrieved 15 June 2018, from https://cs.stanford.edu/~acoates/stl10/
 Benenson, R. (2018). Classification datasets results. Rodrigob.github.io. Retrieved 15 June 2018, from http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html#53544c2d3130
 [duplicate], H. (2018). How to hide axes and gridlines in Matplotlib (python). Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/45148704/howtohideaxesandgridlinesinmatplotlibpython
 VanderPlas, J. (2018). Multiple Subplots  Python Data Science Handbook. Jakevdp.github.io. Retrieved 16 June 2018, from https://jakevdp.github.io/PythonDataScienceHandbook/04.08multiplesubplots.html
 matplotlib.pyplot.subplot — Matplotlib 2.2.2 documentation. (2018). Matplotlib.org. Retrieved 16 June 2018, from https://matplotlib.org/api/_as_gen/matplotlib.pyplot.subplot.html
 matplotlib, m. (2018). more than 9 subplots in matplotlib. Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/4158367/morethan9subplotsinmatplotlib
 pylab_examples example code: subplots_demo.py — Matplotlib 2.0.0 documentation. (2018). Matplotlib.org. Retrieved 16 June 2018, from https://matplotlib.org/2.0.0/examples/pylab_examples/subplots_demo.html
 matplotlib.pyplot.hist — Matplotlib 2.2.2 documentation. (2018). Matplotlib.org. Retrieved 16 June 2018, from https://matplotlib.org/api/_as_gen/matplotlib.pyplot.hist.html
 Python?, I. (2018). Is there a clean way to generate a line histogram chart in Python?. Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/27872723/isthereacleanwaytogeneratealinehistogramchartinpython
 Pyplot Text — Matplotlib 2.2.2 documentation. (2018). Matplotlib.org. Retrieved 16 June 2018, from https://matplotlib.org/gallery/pyplots/pyplot_text.html#sphxglrgallerypyplotspyplottextpy
 [ Google / ICLR 2017 / Paper Summary ] Gradients of Counterfactuals. (2018). Towards Data Science. Retrieved 16 June 2018, from https://towardsdatascience.com/googleiclr2017papersummarygradientsofcounterfactuals6306510935f2
 numpy.transpose — NumPy v1.14 Manual. (2018). Docs.scipy.org. Retrieved 16 June 2018, from https://docs.scipy.org/doc/numpy1.14.0/reference/generated/numpy.transpose.html
 command, G. (2018). Git add and commit in one command. Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/4298960/gitaddandcommitinonecommand
 How to slice an image into red, g. (2018). How to slice an image into red, green and blue channels with misc.imread. Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/37431599/howtosliceanimageintoredgreenandbluechannelswithmiscimread
 How to install Bash shell commandline tool on Windows 10. (2016). Windows Central. Retrieved 16 June 2018, from https://www.windowscentral.com/howinstallbashshellcommandlinewindows10
 Google Colab Free GPU Tutorial — Deep Learning Turkey — Medium. (2018). Medium. Retrieved 16 June 2018, from https://medium.com/deeplearningturkey/googlecolabfreegpututoriale113627b9f5d
 Installation — imgaug 0.2.5 documentation. (2018). Imgaug.readthedocs.io. Retrieved 16 June 2018, from http://imgaug.readthedocs.io/en/latest/source/installation.html
 numpy.absolute — NumPy v1.14 Manual. (2018). Docs.scipy.org. Retrieved 16 June 2018, from https://docs.scipy.org/doc/numpy1.14.0/reference/generated/numpy.absolute.html
 numpy.clip — NumPy v1.10 Manual. (2018). Docs.scipy.org. Retrieved 16 June 2018, from https://docs.scipy.org/doc/numpy1.10.0/reference/generated/numpy.clip.html
 numpy.percentile — NumPy v1.14 Manual. (2018). Docs.scipy.org. Retrieved 16 June 2018, from https://docs.scipy.org/doc/numpy/reference/generated/numpy.percentile.html
 CIFAR10 and CIFAR100 datasets. (2018). Cs.toronto.edu. Retrieved 16 June 2018, from https://www.cs.toronto.edu/~kriz/cifar.html
 [ ICLR 2015 ] Striving for Simplicity: The All Convolutional Net with Interactive Code [ Manual…. (2018). Towards Data Science. Retrieved 16 June 2018, from https://towardsdatascience.com/iclr2015strivingforsimplicitytheallconvolutionalnetwithinteractivecodemanualb4976e206760
 EN10/CIFAR. (2018). GitHub. Retrieved 16 June 2018, from https://github.com/EN10/CIFAR
 Axiomatic Attribution for Deep Networks — — Mukund Sundararajan, Ankur Taly, Qiqi Yan. (2017). Vimeo. Retrieved 16 June 2018, from https://vimeo.com/238242575
 (2018). Arxiv.org. Retrieved 16 June 2018, from https://arxiv.org/pdf/1506.06579.pdf
 Riemann sum. (2018). En.wikipedia.org. Retrieved 16 June 2018, from https://en.wikipedia.org/wiki/Riemann_sum
Bio: Jae Duk Seo is a fourth year computer scientist at Ryerson University.
Original. Reposted with permission.
Related:
 Building Convolutional Neural Network using NumPy from Scratch
 How I Used CNNs and Tensorflow and Lost a Silver Medal in Kaggle Challenge
 Using Tensorflow Object Detection to do Pixel Wise Classification
Pages: 1 2
Top Stories Past 30 Days

