Facebook Open Sourced New Frameworks to Advance Deep Learning Research

I recently started a new newsletter focus on AI education. TheSequence is a no-BS (meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:

The research divisions of technology giants such as Microsoft, Google, Facebook, Amazon, Uber etc. have become some of the most active contributors to open source frameworks in the artificial intelligence (AI) space. Their contributions combine stacks that have been tested at scale in their internal solutions as well as some very advanced ideas from their research labs. While having the biggest tech firms in the world actively contributing to open source deep learning is certainly exciting, it makes it a bit hard for data scientists to keep up with the new developments in the space. Today, I would like to cover three new releases that Facebook AI Research (FAIR) open source in the last month. One month into 2020, three new open source releases from the FAIR team. How is that for a fast growing trend?

FAIR has been a regular contributor to both research and open source frameworks in the deep learning space. From PyTorch to ONNX, the FAIR team has produced an incredible number of contributions to streamline the implementation of deep learning applications. In the last few weeks, the FAIR portfolio added three new open source frameworks.

Polygames

 
Polygames is an open source research framework to train deep learning agents in strategy games through self-play. Polygames is based on the famous concept of zero-learning which allow agents to master environments by interacting with it without any preset training.

At first glance, Polygames seems similar to other game learning frameworks like Alpha Zero or ELF OpenGo but the FAIR stack brings its own set of contributions to the table. For starters, Polygames support a wider list of strategy games such as Hex, Havannah, Minishogi, Connect6, Minesweeper, Mastermind, EinStein würfelt nicht!, Nogo, and Othello which offer researcher a broader set of environments to test deep learning agents. Additionally, Polygames expands traditional zero-learning concepts with a clever architecture that combines deep neural networks and Monte Carlo Tree Search methods. This architecture allows agents to generalize to a larger number of tasks and environments. One of the unexpected benefits of the Polygames framework is the neuroplasticity created in the agents. Polygames’ models are incremental — the framework comes with a script for adding new layers and channels or increasing kernel width — they’re capable of warm start training, allowing the neural network to grow as it trains.

From a programming model standpoint, Polygames provides a library of included games, as well as a single-file API to implement your own game. The developer experience is based on PyTorch and results relatively straightforward to get started.

The FAIR team achieved several milestones with Polygames including beating top human players at the game of Hex19. Created in the 1940s by the poet and mathematician Piet Hein and the economist and mathematician John Nash, the game of hex challenges some of the traditional human game thinking process. The rules are simple. Black and white fill an empty cell, in turn. Black wins if it connects North and South, White wins if it connects West and East. The game is made more fair by a pie rule: at the second move, the second player can decide to swap colors.

The game is hard because, as a connection game, its reward is based on a global, not a local, criteria.

In a series of experiments, Polygames was able to beat the top human players in the game of Hex. The results are shown in the following figure in which the human player conducts the white pieces. The first image represents the Hex opening. In the second stage of the game, the human (White) seems to win — two solid groups are connected to East and West respectively, and look close to connect each other. However, Polygames was able to revert that situation creating a quite complicated position in the center. That position unfold with Polygames finding a winning combination using one of two possible paths.

Source: https://github.com/facebookincubator/polygames

Polygames has been open sourced in GitHub.

PyTorch3D

 
PyTorch3D is a framework designed to train deep learning agents in 3D environments. Despite the large number of vision intelligence systems that need to operate in real world environments, the tools and frameworks for training these type of agents in 3D settings remained highly constrained. PyTorch3D is a highly modular and optimized library with unique capabilities designed to make 3D deep learning easier with PyTorch. PyTorch3D provides a set of frequently used 3D operators and loss functions for 3D data that are fast and differentiable, as well as a modular differentiable rendering API which enables researchers to import these functions into current state-of-the-art deep learning systems right away.

PyTorch3D leverages several recent milestones in 3D deep learning such as FAIR’s Mesh R-CNN, which achieved full 3D object reconstruction from images of complex interior spaces. The framework also uses Detectron2, highly optimized 2D recognition library to successfully push object understanding to the third dimension. PyTorch3D functions for handling rotations and 3D transformations were also central in creating C3DPO, a novel method for learning associations between images and 3D shapes using less annotated training data.

Some of the key contributions of PyTorch3D include:

• Data structure for storing and manipulating triangle meshes: PyTorch3D stores 3D representations in a data structure known as Meshes. This data structure makes it easy for researchers to quickly transform the underlying mesh data into different views to match operators with the most efficient representation of the data.
• Efficient operations on triangle meshes: PyTorch3D includes a series of operations for optimizing the implementations of several common operators and loss functions for 3D data, supporting heterogeneous batches of inputs. This means that researchers and engineers can import the operators in PyTorch3D for faster experimentation without having to re-create or reimplement the operators from scratch at the start of each new project.
• A differentiable mesh renderer: PyTorch3D includes a modular differentiable renderer. Our implementation consists of composable units, allowing users to easily extend the renderer to support custom lighting or shading effects.

Source: https://github.com/facebookresearch/pytorch3d?fbclid=IwAR3qPs1GDJineVfyG8r9fJSwQ1psZB4Ajn3P2JaVbxGFEx95T5bvhji3Kek

PyTorch3D is now available in GitHub.

HiPlot

 
Exploring high dimensional data is one of the challenges of deep learning applications. HiPlot is an interactive visualization tool to help AI researchers discover correlations and patterns in high-dimensional data using parallel plots and other graphical ways to represent information. HiPlot uses a technique known as parallel plots which are a convenient way to visualize and filter high-dimensional data.

From the functional standpoint, HiPlot provides several advantages with other visualization tools:

• Interactivity. In HiPlot, parallel plots are interactive, which makes it easy to change the visualization for different use cases. For example, you can focus on experiments that take a range or value along one or several axes, set the color scheme according to yet another axis, reorder or remove axes, or extract a particular selection of data.
• Simplicity: Using HiPlots only requires a few lines of code. Through a server with the “hiplot” command. You can then access it through a given URL and use it to visualize, manage, and share your experiments. .
• Visualization of Population Based Training: HiPlot provides an easy way to visualize population based training experiments in an XY plot with edges between the different data points. This type of visualization is super common in deep learning experiments.

Source: https://github.com/facebookresearch/hiplot

HiPlot is now available in GitHub.

The FAIR team at Facebook continues innovating across several areas of the deep learning spectrum and actively contributing to the open source community. PyTorch3B, Polygames and HiPlot are FAIR’s latest contributions targeted to streamline the implementation of deep learning applications.

 
Original. Reposted with permission.

Related:

• Microsoft and Google Open Sourced These Frameworks Based on Their Work Scaling Deep Learning Training
• Uber Open Sources the Third Release of Ludwig, its Code-Free Machine Learning Platform
• Getting Started in AI Research