TPOT: A Python Tool for Automating Data Science
TPOT is an open-source Python data science automation tool, which operates by optimizing a series of feature preprocessors and models, in order to maximize cross-validation accuracy on data sets.
Feature preprocessing is important
As we’ve seen in the previous two examples, machine learning model performance is also affected by how the features are represented. Feature preprocessing is a step in machine learning pipelines where we reshape the features in a manner that makes the data set easier for models to classify.
For example, if we’re working on a harder version of the “hill” vs. “valley” signal processing problem with noise:
And we apply a “tuned” random forest to the problem:
We’ll again find that the “tuned” random forest averages a disappointing 57.8% cross-validation accuracy.
However, if we preprocess the features—denoising them via Principal Component Analysis (PCA), for example:
We’ll find that the random forest now achieves an average of 94% cross-validation accuracy by applying a simple feature preprocessing step.
Always explore numerous feature representations for your data. Machines learn differently from humans, and a feature representation that makes sense to us may not make sense to the machine.
Automating data science with TPOT
To summarize what we’ve learned so far about effective machine learning system design, we should:
- Always tune the hyperparameters for our models
- Always try out many different models
- Always explore numerous feature representations for our data
We must also consider the following:
- There are thousands of possible hyperparameter configurations for every model
- There are dozens of popular machine learning models
- There are dozens of popular feature preprocessing methods
This is why it can be so tedious to design effective machine learning systems. This is also why my collaborators and I created TPOT, an open source Python tool that intelligently automates the entire process.
If your data set is compatible with scikit-learn, then TPOT will automatically optimize a series of feature preprocessors and models that maximize the cross-validation accuracy on the data set. For example, if we want TPOT to solve the noisy “hill” vs. “valley” classification problem:
(Before running the code below, make sure to install TPOT first.)
Depending on the machine you’re running it on, 10 TPOT generations should take about 5 minutes to complete. During this time, you’re free to browse Hacker News, refill your cup of coffee, or admire the beautiful weather outside. In the meantime, TPOT will handle all of the work for you.
After 5 minutes of optimization, TPOT will discover a pipeline that achieves 96% cross-validation accuracy on the noisy “hill” vs. “valley” problem—better than the hand-designed pipeline we created above!
If we want to see what pipeline TPOT created, TPOT can export the corresponding scikit-learn code for us with the export() command:
which will look something like:
and shows us that a tuned logistic regression is probably the best model for this problem.
We’ve designed TPOT to be an end-to-end automated machine learning system, which can act as a drop-in replacement for any scikit-learn model that you’re currently using in your workflow.
If TPOT sounds like the tool you’ve been looking for, here’s a few links that you may find useful:
And as always, please feel free to get in touch.
You can find all of the code used in this article on my GitHub. Enjoy!
Bio: Dr. Randy Olson is a postdoctoral researcher at the University of Pennsylvania. As a member of Prof. Jason H. Moore's research lab, he studies biologically-inspired AI and its applications to biomedical problems.
Original. Reposted with permission.