TensorFlow: Building Feed-Forward Neural Networks Step-by-Step
This article will take you through all steps required to build a simple feed-forward neural network in TensorFlow by explaining each step in details.
XOR Logic Gate using Feed-Forward Neural Network (FFNN)
The same concepts applied previously will also hold in all neural networks. There are some changes such as adding more layers or more neurons, changing the type of activation function, or using different loss function.
The data used as input are as follows:
The network architecture to be created using TensorFlow as a FFNN with one hidden layer containing two neurons is as follows:
That hidden layer accepts the inputs from the input layer. Based on its weights and biases, its two activation functions will produce two outputs. The outputs of the hidden layer will be regarded the inputs to the output layer which produces the final expected class scores of the input data.
1. import tensorflow 2. import numpy 3. 4. # Preparing training data (inputs-outputs) 5. training_inputs = tensorflow.placeholder(shape=[None, 2], dtype=tensorflow.float32) 6. training_outputs = tensorflow.placeholder(shape=[None, 1], dtype=tensorflow.float32) #Desired outputs for each input 7. 8. """ 9. Hidden layer with two neurons 10. """ 11. 12. # Preparing neural network parameters (weights and bias) using TensorFlow Variables 13. weights_hidden = tensorflow.Variable(tensorflow.truncated_normal(shape=[2, 2], dtype=tensorflow.float32)) 14. bias_hidden = tensorflow.Variable(tensorflow.truncated_normal(shape=[1, 2], dtype=tensorflow.float32)) 15. 16. # Preparing inputs of the activation function 17. af_input_hidden = tensorflow.matmul(training_inputs, weights_hidden) + bias_hidden 18. 19. # Activation function of the output layer neuron 20. hidden_layer_output = tensorflow.nn.sigmoid(af_input_hidden) 21. 22. 23. """ 24. Output layer with one neuron 25. """ 26. 27. # Preparing neural network parameters (weights and bias) using TensorFlow Variables 28. weights_output = tensorflow.Variable(tensorflow.truncated_normal(shape=[2, 1], dtype=tensorflow.float32)) 29. bias_output = tensorflow.Variable(tensorflow.truncated_normal(shape=[1, 1], dtype=tensorflow.float32)) 30. 31. # Preparing inputs of the activation function 32. af_input_output = tensorflow.matmul(hidden_layer_output, weights_output) + bias_output 33. 34. # Activation function of the output layer neuron 35. predictions = tensorflow.nn.sigmoid(af_input_output) 36. 37. 38. #----------------------------------- 39. 40. # Measuring the prediction error of the network after being trained 41. prediction_error = 0.5 * tensorflow.reduce_sum(tensorflow.subtract(predictions, training_outputs) * tensorflow.subtract(predictions, training_inputs)) 42. 43. # Minimizing the prediction error using gradient descent optimizer 44. train_op = tensorflow.train.GradientDescentOptimizer(0.05).minimize(prediction_error) 45. 46. # Creating a TensorFlow Session 47. sess = tensorflow.Session() 48. 49. # Initializing the TensorFlow Variables (weights and bias) 50. sess.run(tensorflow.global_variables_initializer()) 51. 52. # Training data inputs 53. training_inputs_data = [[1.0, 0.0], 54. [1.0, 1.0], 55. [0.0, 1.0], 56. [0.0, 0.0]] 57. 58. # Training data desired outputs 59. training_outputs_data = [[1.0], 60. [1.0], 61. [0.0], 62. [0.0]] 63. 64. # Training loop of the neural network 65. for step in range(10000): 66. op, err, p = sess.run(fetches=[train_op, prediction_error, predictions], 67. feed_dict={training_inputs: training_inputs_data, 68. training_outputs: training_outputs_data}) 69. print(str(step), ": ", err) 70. 71. # Class scores of some testing data 72. print("Expected class scroes : ", sess.run(predictions, feed_dict={training_inputs: training_inputs_data})) 73. 74. # Printing hidden layer weights initially generated using tf.truncated_normal() 75. print("Hidden layer initial weights : ", sess.run(weights_hidden)) 76. 77. # Printing hidden layer bias initially generated using tf.truncated_normal() 78. print("Hidden layer initial weights : ", sess.run(bias_hidden)) 79. 80. # Printing output layer weights initially generated using tf.truncated_normal() 81. print("Output layer initial weights : ", sess.run(weights_output)) 82. 83. # Printing output layer bias initially generated using tf.truncated_normal() 84. print("Output layer initial weights : ", sess.run(bias_output)) 85. 86. # Closing the TensorFlow Session to free resources 87. sess.close()
Here is the expected class scores of the test data and the weights and biases for both hidden and output layer.
Expected class scores : [[ 0.75373638] [ 0.94796741] [ 0.25110725] [ 0.03870015]] Hidden layer weights : [[ 1.68877864 -3.25296354] [ 1.36028981 -1.6849252 ]] Hidden layer weights : [[-1.27290058 2.33101916]] Output layer weights : [[ 2.42446136] [-5.42509556]] Output layer weights : [[ 1.20168602]]
For more information, visit my YouTube channel:
- Install TensorFlow on Windows with CPU support: https://www.youtube.com/watch?v=MsONo20MRVU
- Intro. to deep learning + explaining how to solve XOR using neural network: https://www.youtube.com/watch?v=EjWDFt-2n9k
- Simple neural network example in Arabic: https://www.youtube.com/watch?v=Jv2d9f6QpJ0
- Simple neural network example in English: https://www.youtube.com/watch?v=Ek3OxE8J1bk
Bio: Ahmed Gad received his B.Sc. degree with excellent with honors in information technology from the Faculty of Computers and Information (FCI), Menoufia University, Egypt, in July 2015. For being ranked first in his faculty, he was recommended to work as a teaching assistant in one of the Egyptian institutes in 2015 and then in 2016 to work as a teaching assistant and a researcher in his faculty. His current research interests include deep learning, machine learning, artificial intelligence, digital signal processing, and computer vision.
Original. Reposted with permission.
Related: