Deep Learning on your phone: PyTorch C++ API for use on Mobile Platforms
The PyTorch Deep Learning framework has a C++ API for use on mobile platforms. This article shows an end-to-end demo of how to write a simple C++ application with Deep Learning capabilities using the PyTorch C++ API such that the same code can be built for use on mobile platforms (both Android and iOS).
By Dhruv Matani, software engineer at Meta (Facebook).
PyTorch is a Deep Learning framework for training and running Machine Learning (ML) Models, accelerating the speed from research to production.
The PyTorch C++ API can be used to write compact, performance sensitive code with Deep Learning capabilities to perform ML inference on mobile platforms. For a general introduction on how to deploy a PyTorch model to production, please see this article.
Steps at a high level
- Train a model (or fetch a pre-trained model), and save in the lite-interpreter format. The lite-interpreter format for a PyTorch model makes the model compatible for running on mobile platforms. This step is not covered in this article.
- Download the PyTorch source code, and build PyTorch from source. This is recommended for mobile deployments.
- Create a .cpp file with code to load your model, run inference using forward(), and print the results.
- Build the .cpp file, and link against PyTorch shared object files.
- Run the file and see the output.
Steps in Detail
Download and install PyTorch from source on a Linux machine.
For other platforms, please see this link.
Create source .cpp file
Store the .cpp file in the same folder as your AddTensorsModelOptimized.ptl file. Let’s call the C++ source file PTDemo.cpp. Let’s call this directory /home/$USER/ptdemo/.
Note: We use symbols from the at:: namespace and not the torch:: namespace (which is what is used by tutorials on the PyTorch web-site) since for mobile builds, we won’t be including the full-jit (TorchScript) interpreter. Instead, we will have access only to the lite-interpreter.
Build and Link
Run the application
The command should print the following:
In this article, we saw how to use the x-platform C++ PyTorch API to load a pre-trained lite-interpreter model with inputs and run inference using that model. The C++ code used is platform agnostic and can be built and run on all mobile platforms supported by PyTorch.
- Introduction to PyTorch For Deep Learning
- Getting Started With PyTorch
- Deploy your PyTorch Model to Production
- Contributing to PyTorch: By someone who doesn’t know a ton about PyTorch
- PyTorch Mobile
- PyTorch C++ API
- Build PyTorch from source
- PyTorch iOS 101 Video walk through
Bio: Dhruv Matani is a software engineer at Meta (Facebook), where he leads projects related to PyTorch (Open Source AI Framework). He is an expert on PyTorch internals, PyTorch Mobile, and is a significant contributor to PyTorch. His work is impacting billions of users across the world. He has extensive experience building and scaling infrastructure for the Facebook Data Platform. Of note are contributions to Scuba, a realtime data analytics platform at Facebook, used for rapid product and system insights. He has a M.S. in Computer Science from Stony Brook University.
- How is Machine Learning Beneficial in Mobile App Development?
- Facebook Open Sourced New Frameworks to Advance Deep Learning Research
- Getting Started with PyTorch Lightning