Complete Guide to Build ConvNet HTTP-Based Application using TensorFlow and Flask RESTful Python API
Complete Guide to Build ConvNet HTTP-Based Application using TensorFlow and Flask RESTful Python APIIn this tutorial, a CNN is to be built, and trained and tested against the CIFAR10 dataset. To make the model remotely accessible, a Flask Web application is created using Python to receive an uploaded image and return its classification label using HTTP.
4. Training the CNN
After building the computational graph of the CNN, next is to train it against the previously prepared training data. The training is done according to the following code. The code starts by preparing the path of the dataset and preparing it into a placeholder. Note that the path should be changed to be suitable to your system. Then it calls the previously discussed functions. The predictions of the trained CNN is used to measure the cost of the network which is to be minimized using the gradient descent optimizer. Note: some of the tensors have a name which is helpful for retrieving such tensors later when testing the CNN.
#Nnumber of classes in the dataset. Used to specify number of outputs in the last fully connected layer.
num_datatset_classes = 10
#Number of rows & columns in each input image. The image is expected to be rectangular Used to reshape the images and specify the input tensor shape.
im_dim = 32
#Number of channels in rach input image. Used to reshape the images and specify the input tensor shape.
num_channels = 3
#Directory at which the training binary files of the CIFAR10 dataset are saved.
patches_dir = "C:\\Users\\Dell\\Downloads\\Compressed\\cifar-10-python\\cifar-10-batches-py\\"
#Reading the CIFAR10 training binary files and returning the input data and output labels. Output labels are used to test the CNN prediction accuracy.
dataset_array, dataset_labels = get_dataset_images(dataset_path=patches_dir, im_dim=im_dim, num_channels=num_channels)
print("Size of data : ", dataset_array.shape)
"""
Input tensor to hold the data read above. It is the entry point of the computational graph.
The given name of 'data_tensor' is useful for retreiving it when restoring the trained model graph for testing.
"""
data_tensor = tensorflow.placeholder(tensorflow.float32, shape=[None, im_dim, im_dim, num_channels], name='data_tensor')
"""
Tensor to hold the outputs label.
The name "label_tensor" is used for accessing the tensor when tesing the saved trained model after being restored.
"""
label_tensor = tensorflow.placeholder(tensorflow.float32, shape=[None], name='label_tensor')
#The probability of dropping neurons in the dropout layer. It is given a name for accessing it later.
keep_prop = tensorflow.Variable(initial_value=0.5, name="keep_prop")
#Building the CNN architecure and returning the last layer which is the fully connected layer.
fc_result2 = create_CNN(input_data=data_tensor, num_classes=num_datatset_classes, keep_prop=keep_prop)
"""
Predicitions probabilities of the CNN for each training sample.
Each sample has a probability for each of the 10 classes in the dataset.
Such tensor is given a name for accessing it later.
"""
softmax_propabilities = tensorflow.nn.softmax(fc_result2, name="softmax_probs")
"""
Predicitions labels of the CNN for each training sample.
The input sample is classified as the class of the highest probability.
axis=1 indicates that maximum of values in the second axis is to be returned. This returns that maximum class probability fo each sample.
"""
softmax_predictions = tensorflow.argmax(softmax_propabilities, axis=1)
#Cross entropy of the CNN based on its calculated probabilities.
cross_entropy = tensorflow.nn.softmax_cross_entropy_with_logits(logits=tensorflow.reduce_max(input_tensor=softmax_propabilities, reduction_indices=[1]),
labels=label_tensor)
#Summarizing the cross entropy into a single value (cost) to be minimized by the learning algorithm.
cost = tensorflow.reduce_mean(cross_entropy)
#Minimizng the network cost using the Gradient Descent optimizer with a learning rate is 0.01.
error = tensorflow.train.GradientDescentOptimizer(learning_rate=.01).minimize(cost)
#Creating a new TensorFlow Session to process the computational graph.
sess = tensorflow.Session()
#Wiriting summary of the graph to visualize it using TensorBoard.
tensorflow.summary.FileWriter(logdir="./log/", graph=sess.graph)
#Initializing the variables of the graph.
sess.run(tensorflow.global_variables_initializer())
"""
Because it may be impossible to feed the complete data to the CNN on normal machines, it is recommended to split the data into a number of patches.
A percent of traning samples is used to create each path. Samples for each path can be randomly selected.
"""
num_patches = 5#Number of patches
for patch_num in numpy.arange(num_patches):
print("Patch : ", str(patch_num))
percent = 80 #percent of samples to be included in each path.
#Getting the input-output data of the current path.
shuffled_data, shuffled_labels = get_patch(data=dataset_array, labels=dataset_labels, percent=percent)
#Data required for cnn operation. 1)Input Images, 2)Output Labels, and 3)Dropout probability
cnn_feed_dict = {data_tensor: shuffled_data,
label_tensor: shuffled_labels,
keep_prop: 0.5}
"""
Training the CNN based on the current patch.
CNN error is used as input in the run to minimize it.
SoftMax predictions are returned to compute the classification accuracy.
"""
softmax_predictions_, _ = sess.run([softmax_predictions, error], feed_dict=cnn_feed_dict)
#Calculating number of correctly classified samples.
correct = numpy.array(numpy.where(softmax_predictions_ == shuffled_labels))
correct = correct.size
print("Correct predictions/", str(percent * 50000/100), ' : ', correct)
Rather than feeding the entire training data to the CNN, the data is divided into set of patches and patch by patch will feed the network using a loop. Each patch contains subset of the training data. The patches are returned using the get_patch function. Such function accepts the input data, labels, and percent of samples to be returned from such data. It then returns subset of the data according to the input percent.
def get_patch(data, labels, percent=70): """ Returning patch to train the CNN. :param data: Complete input data after being encoded and reshaped. :param labels: Labels of the entire dataset. :param percent: Percent of samples to get returned in each patch. :return: Subset of the data (patch) to train the CNN model. """ #Using the percent of samples per patch to return the actual number of samples to get returned. num_elements = numpy.uint32(percent*data.shape[0]/100) shuffled_labels = labels#Temporary variable to hold the data after being shuffled. numpy.random.shuffle(shuffled_labels)#Randomly reordering the labels. """ The previously specified percent of the data is returned starting from the beginning until meeting the required number of samples. The labels indices are also used to return their corresponding input images samples. """ return data[shuffled_labels[:num_elements], :, :, :], shuffled_labels[:num_elements]
5. Saving the Trained CNN Model
After training the CNN, the model is saved for reuse later for testing it in another Python script. You should also change the path where the model is saved to be suitable to your system.
#Saving the model after being trained.
saver = tensorflow.train.Saver()
save_model_path = "C:\\model\\"
save_path = saver.save(sess=sess, save_path=save_model_path+"model.ckpt")
print("Model saved in : ", save_path)
6. Preparing the Test Data and Restoring the Trained CNN Model
Before testing the trained model, it is required to prepare the test data and restore the previously trained model. Test data preparation is similar to what happened with the training data except that there is just a single binary file to be decoded. The test file is decoded according to the modified get_dataset_images function. This function calls the unpickle_patch function exactly as what done before with training data.
def get_dataset_images(test_path_path, im_dim=32, num_channels=3):
"""
Similar to the one used in training except that there is just a single testing binary file for testing the CIFAR10 trained models.
"""
print("Working on testing patch")
data_dict = unpickle_patch(test_path_path)
images_data = data_dict[b"data"]
dataset_array = numpy.reshape(images_data, newshape=(len(images_data), im_dim, im_dim, num_channels))
return dataset_array, data_dict[b"labels"]
7. Testing the Trained CNN Model.
After preparing the test data and restoring the trained model, we can start testing the model according to the following code. What worth mentioning is that our goal is to just return the network predictions for the input samples. This is why the TF session runs to return just the predictions. When training the CNN, the session runs to minimize the cost. In testing, we are not interested in minimizing the cost anymore. Another interesting point is that the keep probability of the dropout layer is now set to 1. That means do not drop any node. This is because we are just using the pre-trained model after settling on what nodes to drop. Now we just use what the model did before and not interested in making modification to it by dropping other nodes.
#Dataset path containing the testing binary file to be decoded.
patches_dir = "C:\\Users\\Dell\\Downloads\\Compressed\\cifar-10-python\\cifar-10-batches-py\\"
dataset_array, dataset_labels = get_dataset_images(test_path_path=patches_dir + "test_batch", im_dim=32, num_channels=3)
print("Size of data : ", dataset_array.shape)
sess = tensorflow.Session()
#Restoring the previously saved trained model.
saved_model_path = 'C:\\Users\\Dell\\Desktop\\model\\'
saver = tensorflow.train.import_meta_graph(saved_model_path+'model.ckpt.meta')
saver.restore(sess=sess, save_path=saved_model_path+'model.ckpt')
#Initalizing the varaibales.
sess.run(tensorflow.global_variables_initializer())
graph = tensorflow.get_default_graph()
"""
Restoring previous created tensors in the training phase based on their given tensor names in the training phase.
Some of such tensors will be assigned the testing input data and their outcomes (data_tensor, label_tensor, and keep_prop).
Others are helpful in assessing the model prediction accuracy (softmax_propabilities and softmax_predictions).
"""
softmax_propabilities = graph.get_tensor_by_name(name="softmax_probs:0")
softmax_predictions = tensorflow.argmax(softmax_propabilities, axis=1)
data_tensor = graph.get_tensor_by_name(name="data_tensor:0")
label_tensor = graph.get_tensor_by_name(name="label_tensor:0")
keep_prop = graph.get_tensor_by_name(name="keep_prop:0")
#keep_prop is equal to 1 because there is no more interest to remove neurons in the testing phase.
feed_dict_testing = {data_tensor: dataset_array,
label_tensor: dataset_labels,
keep_prop: 1.0}
#Running the session to predict the outcomes of the testing samples.
softmax_propabilities_, softmax_predictions_ = sess.run([softmax_propabilities, softmax_predictions],
feed_dict=feed_dict_testing)
#Assessing the model accuracy by counting number of correctly classified samples.
correct = numpy.array(numpy.where(softmax_predictions_ == dataset_labels))
correct = correct.size
print("Correct predictions/10,000 : ", correct)
8. Building the Flask Web Application
After training the CNN model, we can add it to a HTTP server and allow users to use it online. User will upload an image using a HTTP client. The uploaded image will be received by the HTTP server or more specifically by a Flask Web application. Such application will predict the class label of the image based on the trained model and finally returns the class label back to the HTTP client. Such discussion is summarized in figure 5.
Figure 5
import flask
#Creating a new Flask Web application. It accepts the package name.
app = flask.Flask("CIFAR10_Flask_Web_App")
"""
To activate the Web server to receive requests, the application must run.
A good practice is to check whether the file is whether the file called from an external Python file or not.
If not, then it will run.
"""
if __name__ == "__main__":
"""
In this example, the app will run based on the following properties:
host: localhost
port: 7777
debug: flag set to True to return debugging information.
"""
app.run(host="localhost", port=7777, debug=True)
Currently, there is no functions the server provide. The first thing the server should do is to allow the user to upload an image. When the user visits the root URL of the application, the application do nothing. The application can redirect the user to an HTML page at which the user could upload an image. To do that, the application has a function called redirect_upload to redirect the user to a page for uploading an image. What lets this function to get executed after the user visits the root of the app is the routing created using the following line:
app.add_url_rule(rule="/", endpoint="homepage", view_func=redirect_upload)
This line says that if the user visits the root of the app (marked as "/"), then the viewer function (redirect_upload) will be called. Such function do nothing except rendering a HTML page called upload_image.html. Such page is located under the special templatesdirectory of the server. A page inside the templates directory is rendered by calling the render_template function. Note that there is an attribute called endpoint which makes it easy to reuse the same route multiple times without hard coding it.
def redirect_upload(): """ A viewer function that redirects the Web application from the root to a HTML page for uploading an image to get classified. The HTML page is located under the /templates directory of the application. :return: HTML page used for uploading an image. It is 'upload_image.html' in this exmaple. """ return flask.render_template(template_name_or_list="upload_image.html") """ Creating a route between the homepage URL (http://localhost:7777) to a viewer function that is called after getting to such URL. Endpoint 'homepage' is used to make the route reusable without hard-coding it later. """ app.add_url_rule(rule="/", endpoint="homepage", view_func=redirect_upload)
The screen of the HTML page rendered is shown in figure 6.
Figure 6
Here is the HTML code of such page. It is a simple form that allows the user to upload an image file. When submitting such form, a POST HTTP message is to be returned to the URL http://localhost:7777/upload/.
<!DOCTYPE html>
<html lang="en">
<head>
<link rel="stylesheet" type="text/css" href="{{url_for(endpoint='static', filename='project_styles.css')}}">
<meta charset="UTF-8">
<title>Upload Image</title>
</head>
<body>
<form enctype="multipart/form-data" method="post" action="http://localhost:7777/upload/">
<center>
<h3>Select CIFAR10 image to predict its label.</h3>
<input type="file" name="image_file" accept="image/*"><br>
<input type="submit" value="Upload">
</center>
</form>
</body>
</html>
After returning back to the server from the HTML form, the viewer function that is associated with the URL specified in the form action attribute will be called which is the upload_image function. Such function gets the image selected by the user and saves it to the server.
def upload_image():
"""
Viewer function that is called in response to getting to the 'http://localhost:7777/upload' URL.
It uploads the selected image to the server.
:return: redirects the application to a new page for predicting the class of the image.
"""
#Global variable to hold the name of the image file for reuse later in prediction by the 'CNN_predict' viewer functions.
global secure_filename
if flask.request.method == "POST":#Checking of the HTTP method initiating the request is POST.
img_file = flask.request.files["image_file"]#Getting the file name to get uploaded.
secure_filename = werkzeug.secure_filename(img_file.filename)#Getting a secure file name. It is a good practice to use it.
img_path = os.path.join(app.root_path, secure_filename)#Preparing the full path under which the image will get saved.
img_file.save(img_path)#Saving the image in the specified path.
print("Image uploaded successfully.")
"""
After uploading the image file successfully, next is to predict the class label of it.
The application will fetch the URL that is tied to the HTML page responsible for prediction and redirects the browser to it.
The URL is fetched using the endpoint 'predict'.
"""
return flask.redirect(flask.url_for(endpoint="predict"))
return "Image upload failed."
"""
Creating a route between the URL (http://localhost:7777/upload) to a viewer function that is called after navigating to such URL.
Endpoint 'upload' is used to make the route reusable without hard-coding it later.
The set of HTTP method the viewer function is to respond to is added using the 'methods' argument.
In this case, the function will just respond to requests of method of type POST.
"""
app.add_url_rule(rule="/upload/", endpoint="upload", view_func=upload_image, methods=["POST"])
After uploading the image successfully to the server, we are ready to read the image and predict its class label using the previously trained CNN model. For such reason, the upload_image function redirects the application to the viewer function that is responsible for predicting the class label of an image. Such viewer function is reached by its endpoint as specified in this line:
return flask.redirect(flask.url_for(endpoint="predict"))
The method associated with endpoint="predict" will be called which is the CNN_predict function. Such method reads the image and checks whether it matches the dimensions of the CIFAR-10 dataset which is 32x32x3. If the image matches the specifications of the CIFAR-10 dataset, then it will be passed to a function responsible for making prediction as in the following line:
predicted_class = CIFAR10_CNN_Predict_Image.main(img)
The main function responsible for predicting the class label of an image is defined as shown below. It restores the trained model and runs a session that returns the predicted class of the image. The predicted class is returned back to the Flask Web application.
def CNN_predict(): """ Reads the uploaded image file and predicts its label using the saved pre-trained CNN model. :return: Either an error if the image is not for CIFAR10 dataset or redirects the browser to a new page to show the prediction result if no error occurred. """ """ Setting the previously created 'secure_filename' to global. This is because to be able invoke a global variable created in another function, it must be defined global in the caller function. """ global secure_filename #Reading the image file from the path it was saved in previously. img = scipy.misc.imread(os.path.join(app.root_path, secure_filename)) """ Checking whether the image dimensions match the CIFAR10 specifications. CIFAR10 images are RGB (i.e. they have 3 dimensions). It number of dimenions was not equal to 3, then a message will be returned. """ if(img.ndim) == 3: """ Checking if the number of rows and columns of the read image matched CIFAR10 (32 rows and 32 columns). """ if img.shape[0] == img.shape[1] and img.shape[0] == 32: """ Checking whether the last dimension of the image has just 3 channels (Red, Green, and Blue). """ if img.shape[-1] == 3: """ Passing all conditions above, the image is proved to be of CIFAR10. This is why it is passed to the predictor. """ predicted_class = CIFAR10_CNN_Predict_Image.main(img) """ After predicting the class label of the input image, the prediction label is rendered on an HTML page. The HTML page is fetched from the /templates directory. The HTML page accepts an input which is the predicted class. """ return flask.render_template(template_name_or_list="prediction_result.html", predicted_class=predicted_class) else: # If the image dimensions do not match the CIFAR10 specifications, then an HTML page is rendered to show the problem. return flask.render_template(template_name_or_list="error.html", img_shape=img.shape) else: # If the image dimensions do not match the CIFAR10 specifications, then an HTML page is rendered to show the problem. return flask.render_template(template_name_or_list="error.html", img_shape=img.shape) return "An error occurred."#Returned if there is a different error other than wrong image dimensions. """ Creating a route between the URL (http://localhost:7777/predict) to a viewer function that is called after navigating to such URL. Endpoint 'predict' is used to make the route reusable without hard-coding it later. """ app.add_url_rule(rule="/predict/", endpoint="predict", view_func=CNN_predict)
The main function responsible for predicting the class label of an image is defined as shown below. It restores the trained model and runs a session that returns the predicted class of the image. The predicted class is returned back to the Flask Web application.
def main(img):
"""
The 'main' method accepts an input image array of size 32x32x3 and returns its class label.
:param img:RGB image of size 32x32x3.
:return:Predicted class label.
"""
#Dataset path containing a binary file with the labels of classes. Useful to decode the prediction code into a significant textual label.
patches_dir = "C:\\cifar-10-python\\cifar-10-batches-py\\"
dataset_array = numpy.random.rand(1, 32, 32, 3)
dataset_array[0, :, :, :] = img
sess = tensorflow.Session()
#Restoring the previously saved trained model.
saved_model_path = 'C:\\model\\'
saver = tensorflow.train.import_meta_graph(saved_model_path+'model.ckpt.meta')
saver.restore(sess=sess, save_path=saved_model_path+'model.ckpt')
#Initalizing the varaibales.
sess.run(tensorflow.global_variables_initializer())
graph = tensorflow.get_default_graph()
"""
Restoring previous created tensors in the training phase based on their given tensor names in the training phase.
Some of such tensors will be assigned the testing input data and their outcomes (data_tensor, label_tensor, and keep_prop).
Others are helpful in assessing the model prediction accuracy (softmax_propabilities and softmax_predictions).
"""
softmax_propabilities = graph.get_tensor_by_name(name="softmax_probs:0")
softmax_predictions = tensorflow.argmax(softmax_propabilities, axis=1)
data_tensor = graph.get_tensor_by_name(name="data_tensor:0")
label_tensor = graph.get_tensor_by_name(name="label_tensor:0")
keep_prop = graph.get_tensor_by_name(name="keep_prop:0")
#keep_prop is equal to 1 because there is no more interest to remove neurons in the testing phase.
feed_dict_testing = {data_tensor: dataset_array,
keep_prop: 1.0}
#Running the session to predict the outcomes of the testing samples.
softmax_propabilities_, softmax_predictions_ = sess.run([softmax_propabilities, softmax_predictions],
feed_dict=feed_dict_testing)
label_names_dict = unpickle_patch(patches_dir + "batches.meta")
dataset_label_names = label_names_dict[b"label_names"]
return dataset_label_names[softmax_predictions_[0]].decode('utf-8')
The returned class label of the image will be rendered on a new HTML page named prediction_result.html as instructed by the CNN_predict function in this line as in figure 7.
Figure 7
Note that the Flask app uses the Jinja2 template engine that allows the HTML page to accept input arguments. The input argument passed in this case is predicted_class=predicted_class.
return flask.render_template(template_name_or_list="prediction_result.html", predicted_class=predicted_class)
The HTML code of such page is as follows.
<!DOCTYPE html>
<html lang="en">
<head>
<link rel="stylesheet" type="text/css" href="{{url_for(endpoint='static', filename='project_styles.css')}}">
<script type="text/javascript" src="{{url_for(endpoint='static', filename='result.js')}}"></script>
<meta charset="UTF-8">
<title>Prediction Result</title>
</head>
<body onload="show_alert('{{predicted_class}}')">
<center><h1>Predicted Class Label : <span>{{predicted_class}}</span></h1>
<br>
<a href="{{url_for(endpoint='homepage')}}"><span>Return to homepage</span>.</a>
</center>
</body>
</html>
It is a template that is filled by the predicted class of the image which is passes as an argument to the HTML page as in this part of the code:
<span>{{predicted_class}}</span>
For more information about the Flask RESTful API, you can visit such tutorial https://www.tutorialspoint.com/flask/index.htm.
The complete project is available at Github in this link: https://github.com/ahmedfgad/CIFAR10CNNFlask
Bio: Ahmed Gad received his B.Sc. degree with excellent with honors in information technology from the Faculty of Computers and Information (FCI), Menoufia University, Egypt, in July 2015. For being ranked first in his faculty, he was recommended to work as a teaching assistant in one of the Egyptian institutes in 2015 and then in 2016 to work as a teaching assistant and a researcher in his faculty. His current research interests include deep learning, machine learning, artificial intelligence, digital signal processing, and computer vision.
Original. Reposted with permission.
Related:
- Building Convolutional Neural Network using NumPy from Scratch
- Derivation of Convolutional Neural Network from Fully Connected Network Step-By-Step
- Introduction to Optimization with Genetic Algorithm
