Deploying scikit-learn Models at Scale

Find out how to serve your scikit-learn model in an auto-scaling, serverless environment! Today, we’ll take a trained scikit-learn model and deploy it on Cloud ML Engine.

By Yufeng Guo, Developer and Machine Learning Advocate @ Google Cloud


Scikit-learn is great for putting together a quick model to test out your dataset. But what if you want to run it against incoming live data? Find out how to serve your scikit-learn model in an auto-scaling, serverless environment!

Cloud AI Adventures: explore the art, science, and tools of machine learning.

Today, we’ll take a trained scikit-learn model and deploy it on Cloud ML Engine. Use this post to set up your own deployment pipeline, and you’ll never need to worry about how to deploy and scale your models again!


Let’s say you have a zoo…

Say you have a model you’ve trained up using a scikit-learn model, and now you want to set up a prediction server. Let’s see how to do this based on our code we had in a previous episode about zoo animals.

To export the model, we’ll use the joblib library from sklearn.externals.

from sklearn.externals import joblib
joblib.dump(clf, 'model.joblib')

We can use joblib.dump() to export the model to a file. We’ll call ours model.joblib.

Once we commit and run this kernel, we’ll be able to retrieve the output from the kernel:

model.joblib — ready for download

With our trained scikit-learn model in hand, we are ready to head over to Google Cloud ML Engine to load up the model to serve predictions.

That’s right, we can get all the auto-scaling, secured REST API goodness for not only TensorFlow, but also for scikit-learn (and XGBoost)! This enables you to easily transition back and forth between scikit-learn and TensorFlow.


Serve up some pie…and predictions

The first step to getting our model in the cloud is to upload the model.joblibfile to Google Cloud Storage.

Notice my model.joblib file is wrapped in a folder called zoo_model

Organization tip: It is required that the name of the file be literally “model.joblib”, so you’ll probably want to stick the file inside a folder with a name you’ll remember. Otherwise, later on when you create more models , they’ll all be the same name of model.joblib!

Let’s create our model and version, specifying that we are loading up a scikit-learn model, and select the runtime version of Cloud ML engine, as well as the version of Python that we used to export this model. Since we were running our training on Kaggle, that’s Python 3.

Give it a bit of time to setup…. And that’s basically it! We now have a scikit-learn model serving in the cloud!


Who you gonna call? Cloud ML Predictions!

Now of course, a scalable model is no use without being able to call those predictions. Let’s take a look at how simple that is. I’ve gone ahead and pulled one sample row from our data, whose answer should be category “4”. We’ll present the data to Cloud ML Engine as a simple array, encoded as a json file.


Here I’m taking the test features dataframe and extracting row 10 row from it, and then calling .values to get the underlying numpy array. Unfortunately, numpy arrays don’t print with commas between their values, and we really want the commas, so I’ve turned the numpy array into a Python list and printed that out. (Yes, you could, and probably should really just use the jsonlibrary and encode it properly rather than relying on print behavior!)

I’ve saved the array to an input file, and now we can call the prediction REST API for our scikit-learn model. gcloud has a built-in utility to do this. We are expecting a response of 4, which is indeed what we get back! Huzzah!


Now what?

You can follow the steps in this video to deploy your scikit-learn model to production, or step it up by turning it into automated pipeline so that each time you make a new model, it gets deployed out so you can test it! Head on over to Cloud ML Engine and upload your scikit-learn model to get auto-scaling predictions at your fingertips!

For a more detailed treatment on this topic, the documentation has some great guides:

Thanks for reading this episode of Cloud AI Adventures. If you’re enjoying the series, please let me know by clapping for the article. If you want more machine learning action, be sure to follow me on Medium or subscribe to the YouTube channel to catch future episodes as they come out. More episodes coming at you soon!

Bio: Yufeng Guo is a Developer and Machine Learning Advocate @ Google Cloud. He is also a runner, chef, and musician. Opinions expressed herein are solely his own.

Original. Reposted with permission.