Inside the Mind of a Neural Network with Interactive Code in Tensorflow

Interior Gradients / Positive and Negative Attributes

 

alpha value of 1.0

Now lets use interior gradient, with the alpha values of ….
0.01, 0.01, 0.03, 0.04, 0.1, 0.5, 0.6, 0.7, 0.8 and 1.0 (same as original input), to visualize the the inner workings of the network.

All of the images above represents the gradient respect to the input with different alpha values. We can observe as alpha value increases the gradients becomes closer to 0 (black) due to the saturation within the network. We can clearly see the difference of how gradients change when we view all of the images in a gif format.

However, I was able to make an interesting observation, as alpha value becomes smaller my network wasn’t able to make the correct predictions. As seen below when the alpha value is small (0.01) the network predict mostly 3 or 4. (Cat/Deer)

Red Box → Models Prediction

But as alpha value increased, eventually ending up at 1.0, we can observe that the accuracy of the model increases. Finally, lets take a look at positive and negative attributes from the gradient at each alpha values.

Blue Pixel → Positive Attributes Overlay-ed with Original Image
Red Pixel → Negative Attributes Overlay-ed with the Original Image

Again, it is easier to view the change if we create a gif out of the images.

 

Integral Gradient / Positive and Negative Attributes

 

Left Image → Integral Gradient with Step Size 3000 for riemann approximation
Right Image → Blue Color for Positive Attributes and Red Color for Negative Attributes

Finally when we use Integral Gradient to visualize the gradients we can observe something like above. One very interesting fact I was able to find was how the network predicted horse (7), dog (5) and deer (4).

Lets first take a look at the original images, as seen above the left, middle images are horses and right image is a sheep.

When we visualize the Integral Gradient we can see something interesting. As seen in the middle image the network thought the head portion of a horse was most important. However, when only taking the head, the network thought it was an image of a dog. (And it kinda looks like a dog). Similar story for the right image, when the network only taken into account of only the head of a sheep, it predicts as seeing a dog.

However, as seen on the most left image, when the network takes in the overall shape of a horse (including some portion of a human) it correctly identifies the image as a horse. AMAZING!

 

Interactive Code

 

For Google Colab, you would need a google account to view the codes, also you can’t run read only scripts in Google Colab so make a copy on your play ground. Finally, I will never ask for permission to access your files on Google Drive, just FYI. Happy Coding! Also for transparency I uploaded all of the training logs on my github.

To access the code for this post please click here, for training logs click here.

 

Final Words

 
As this area is still growing and being developed there will be new methods and findings. (I hope I can contribute to those findings as well.). Finally, if you want to access the code from the original authors of the paper “Axiomatic Attribution for Deep Networks” please click here.

If any errors are found, please email me at jae.duk.seo@gmail.com, if you wish to see the list of all of my writing please view my website here.

Meanwhile follow me on my twitter here, and visit my website, or my Youtube channel for more content. I also implemented Wide Residual Networks, please click here to view the blog post.

 
Reference

1. http://Axiom/definition/theorem/proof. (2018). YouTube. Retrieved 14 June 2018, from https://www.youtube.com/watch?v=OeC5WuZbNMI
2. axiom — Dictionary Definition. (2018). Vocabulary.com. Retrieved 14 June 2018, from https://www.vocabulary.com/dictionary/axiom
3. Axiomatic Attribution for Deep Networks — — Mukund Sundararajan, Ankur Taly, Qiqi Yan. (2017). Vimeo. Retrieved 14 June 2018, from https://vimeo.com/238242575
4. STL-10 dataset. (2018). Cs.stanford.edu. Retrieved 15 June 2018, from https://cs.stanford.edu/~acoates/stl10/
5. Benenson, R. (2018). Classification datasets results. Rodrigob.github.io. Retrieved 15 June 2018, from http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html#53544c2d3130
6. [duplicate], H. (2018). How to hide axes and gridlines in Matplotlib (python). Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/45148704/how-to-hide-axes-and-gridlines-in-matplotlib-python
7. VanderPlas, J. (2018). Multiple Subplots | Python Data Science Handbook. Jakevdp.github.io. Retrieved 16 June 2018, from https://jakevdp.github.io/PythonDataScienceHandbook/04.08-multiple-subplots.html
8. matplotlib.pyplot.subplot — Matplotlib 2.2.2 documentation. (2018). Matplotlib.org. Retrieved 16 June 2018, from https://matplotlib.org/api/_as_gen/matplotlib.pyplot.subplot.html
9. matplotlib, m. (2018). more than 9 subplots in matplotlib. Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/4158367/more-than-9-subplots-in-matplotlib
10. pylab_examples example code: subplots_demo.py — Matplotlib 2.0.0 documentation. (2018). Matplotlib.org. Retrieved 16 June 2018, from https://matplotlib.org/2.0.0/examples/pylab_examples/subplots_demo.html
11. matplotlib.pyplot.hist — Matplotlib 2.2.2 documentation. (2018). Matplotlib.org. Retrieved 16 June 2018, from https://matplotlib.org/api/_as_gen/matplotlib.pyplot.hist.html
12. Python?, I. (2018). Is there a clean way to generate a line histogram chart in Python?. Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/27872723/is-there-a-clean-way-to-generate-a-line-histogram-chart-in-python
13. Pyplot Text — Matplotlib 2.2.2 documentation. (2018). Matplotlib.org. Retrieved 16 June 2018, from https://matplotlib.org/gallery/pyplots/pyplot_text.html#sphx-glr-gallery-pyplots-pyplot-text-py
14. [ Google / ICLR 2017 / Paper Summary ] Gradients of Counterfactuals. (2018). Towards Data Science. Retrieved 16 June 2018, from https://towardsdatascience.com/google-iclr-2017-paper-summary-gradients-of-counterfactuals-6306510935f2
15. numpy.transpose — NumPy v1.14 Manual. (2018). Docs.scipy.org. Retrieved 16 June 2018, from https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.transpose.html
16. command, G. (2018). Git add and commit in one command. Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/4298960/git-add-and-commit-in-one-command
17. How to slice an image into red, g. (2018). How to slice an image into red, green and blue channels with misc.imread. Stack Overflow. Retrieved 16 June 2018, from https://stackoverflow.com/questions/37431599/how-to-slice-an-image-into-red-green-and-blue-channels-with-misc-imread
18. How to install Bash shell command-line tool on Windows 10. (2016). Windows Central. Retrieved 16 June 2018, from https://www.windowscentral.com/how-install-bash-shell-command-line-windows-10
19. Google Colab Free GPU Tutorial — Deep Learning Turkey — Medium. (2018). Medium. Retrieved 16 June 2018, from https://medium.com/deep-learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d
20. Installation — imgaug 0.2.5 documentation. (2018). Imgaug.readthedocs.io. Retrieved 16 June 2018, from http://imgaug.readthedocs.io/en/latest/source/installation.html
21. numpy.absolute — NumPy v1.14 Manual. (2018). Docs.scipy.org. Retrieved 16 June 2018, from https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.absolute.html
22. numpy.clip — NumPy v1.10 Manual. (2018). Docs.scipy.org. Retrieved 16 June 2018, from https://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.clip.html
23. numpy.percentile — NumPy v1.14 Manual. (2018). Docs.scipy.org. Retrieved 16 June 2018, from https://docs.scipy.org/doc/numpy/reference/generated/numpy.percentile.html
24. CIFAR-10 and CIFAR-100 datasets. (2018). Cs.toronto.edu. Retrieved 16 June 2018, from https://www.cs.toronto.edu/~kriz/cifar.html
25. [ ICLR 2015 ] Striving for Simplicity: The All Convolutional Net with Interactive Code [ Manual…. (2018). Towards Data Science. Retrieved 16 June 2018, from https://towardsdatascience.com/iclr-2015-striving-for-simplicity-the-all-convolutional-net-with-interactive-code-manual-b4976e206760
26. EN10/CIFAR. (2018). GitHub. Retrieved 16 June 2018, from https://github.com/EN10/CIFAR
27. Axiomatic Attribution for Deep Networks — — Mukund Sundararajan, Ankur Taly, Qiqi Yan. (2017). Vimeo. Retrieved 16 June 2018, from https://vimeo.com/238242575
28. (2018). Arxiv.org. Retrieved 16 June 2018, from https://arxiv.org/pdf/1506.06579.pdf
29. Riemann sum. (2018). En.wikipedia.org. Retrieved 16 June 2018, from https://en.wikipedia.org/wiki/Riemann_sum

 
Bio: Jae Duk Seo is a fourth year computer scientist at Ryerson University.

Original. Reposted with permission.

Related:

• Building Convolutional Neural Network using NumPy from Scratch
• How I Used CNNs and Tensorflow and Lost a Silver Medal in Kaggle Challenge
• Using Tensorflow Object Detection to do Pixel Wise Classification