Silver BlogFacebook Has Been Quietly Open Sourcing Some Amazing Deep Learning Capabilities for PyTorch

The new release of PyTorch includes some impressive open source projects for deep learning researchers and developers.

Header image


PyTorch has become one of the most popular deep learning frameworks in the market and certainly a favorite of the research community when comes to experimentation. As a reference, PyTorch citations in papers on ArXiv grew 194 percent in the first half of 2019 alone, as noted by O’Reilly. For years, Facebook has based its deep learning work in a combination of PyTorch and Caffe2 and has put a lot of resources to support the PyTorch stack and developer community. Yesterday, Facebook released the latest version of PyTorch which showcases some state-of-the-art deep learning capabilities.

There have been plenty of articles covering the launch of PyTorch 1.3. Instead of doing that, I would like to focus on some of the new projects accompanying the new release of the deep learning framework. Arguably, the most impressive capability of PyTorch is how quickly it has been able to incorporate implementations about new research technique. Not surprisingly, the artificial intelligence(AI) research community has started adopting PyTorch as one of the preferred stacks to experiment with new deep learning methods. The new release of PyTorch continues this trend by adding some impressive open source projects surrounding the core stack.


Secure Machine Learning Research with Crypten

Data privacy is one of the big challenges of modern machine learning applications. In order to build machine learning models, researchers need to have complete access to datasets that often contain sensitive data. The idea of models that work effectively with encrypted datasets has long been an elusive goal of the machine learning space. While research in areas such as homomorphic encryption or secure, multi-party computation has been rapidly advancing, its adoption in machine learning stacks remains limited at best.

Crypten is a new, easy-to-use software framework built on PyTorch to facilitate research in secure and privacy-preserving machine learning. Crypten incorporates security and data privacy techniques as a native citizen of machine learning models allowing researchers to leverage these methods without having to become an expert in cryptography.

The core of Crypten’s architecture is based on an implementation of secure multi-party computations. This technique enables computations such as machine learning models to be performed over encrypted datasets. In the case of PyTorch, the use of Crypten is illustrated in the following figure:

Compared to other privacy libraries in the space, Crypten brings some tangible benefits.

  1. PyTorch-Based: Developers using Crypten have access to the entire PyTorch stack. Also, Crypten has been optimized for machine learning scenarios and doesn’t require any special adaptations.
  2. Library-Based: Crypten is implemented as a native PyTorch library as not as a compiler like most privacy frameworks in the market.
  3. Real World Machine Learning: Crypten was built to address privacy in real world machine learning scenarios. The framework supports privacy across different structures ranging. from basic linear models to complex neural network architectures.


Modular Object Detection with Detectron2

Since its release in 2018, the Detectron object detection platform has become one of Facebook AI Research (FAIR)’s most widely adopted open source projects. Detectron2 brings a series of new research and production capabilities to the popular framework. While the original Detectron was written in Caffe2, Detectron2 represents a rewrite of the original framework in PyTorch and brings some exciting object detection capabilities.

Detectron2 was built to enable object detection at large scale. The Framework is used to rapidly design and train the next-generation pose detection models that power Smart Camera, the AI camera system in Facebook’s Portal video-calling devices. From an implementation and capability standpoint, Detectron2 brings some tangible improvements over its predecessor:

  • Modularity: Detectron2, introduces a modular design that allows users to plug custom module implementations into almost any part of an object detection system.
  • New Models: Detectron2 includes all the models that were available in the original Detectron but also features several new models, including Cascade R-CNN, Panoptic FPN, and TensorMask.
  • New Tasks: Detectron2 complements its object and pose detection capabilities with new tasks such as semantic segmentation and panoptic segmentation, a task that combines both semantic and instance segmentation.
  • Detectron2go: Detectron2 includes the Detectron2go module to make it easier to deploy advanced new models to production. These features include standard training workflows with in-house data sets, network quantization, and model conversion to optimized formats for cloud and mobile deployment.


Better Model Interpretability Using Captum

Model interpretability remains one of the biggest challenges of modern machine learning. Captum is a flexible, and easy-to-use model interpretability library for PyTorch. It makes state-of-the-art algorithms for interpretability readily available to developers and researchers.

Using Captum, PyTorch researchers can quickly evaluate and benchmark their models against other algorithms available in the library. Developers can also use Captum to improve and troubleshoot models by facilitating the identification of different features that contribute to a model’s output in order to design better models and troubleshoot unexpected model outputs.

Captum includes a large portfolio of interpretability algorithms which can be categorized using three main groups:

  • General Attribution: Evaluates contribution of each input feature to the output of a model.
  • Layer Attribution: Evaluates contribution of each neuron in a given layer to the output of the model.
  • Neuron Attribution: Evaluates contribution of each input feature on the activation of a particular hidden neuron.

Captum complements its programmable capabilities with Captum Insights, an interpretability visualization widget built on top of Captum to facilitate model understanding. Captum Insights works across images, text, and other features to help users understand feature attribution.

The new release of PyTorch goes beyond improving the core capabilities of the framework and bring state-of-the-art research to deep learning developers. Projects like Captum, Detectron2 and Crypten complement the core PyTorch stack and helps to bridge the gap between research and production deep learning systems.

Original. Reposted with permission.