The Machine Learning Research Championed by the Biggest AI Labs in the World
How Google, Microsoft, Facebook, DeepMind, OpenAI, Amazon and IBM think about the future of AI.
I recently started a new newsletter focus on AI education and already has over 50,000 subscribers. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:
Recently, one of my students asked me a question as of whether DeepMind was solely working in reinforcement learning applications. The answer is obviously no but the question is still valid as it rooted in the fact that most of DeepMind’s highly publicized work such as AlphaGo, MuZero or AlphaFold are based in reinforcement learning. The fascinating thing is that you can extrapolate the same point to most of the biggest artificial intelligence(AI) labs in the world.
While companies like Google, Facebook or Microsoft are certainly working across all areas of deep learning, they also have become champions of specific methods and disciplines. I find that movement kind of fascinating because, in some ways, tells us which techniques the biggest AI labs in the world believe is key to the near future of AI. Without trying to be exclusive, here is some of the most intriguing new areas of deep learning championed by some of the biggest AI-tech companies in the world.
DeepMind — Reinforcement Learning: Without a doubt, DeepMind is synonym with reinforcement learning. The Alphabet subsidiary has championed major breakthroughs in this area such as AlphaGo, MuZero and the recent AlphaFold.
Facebook — Self-Supervised Learning: Under Turin Award winner Yann LeCun, Facebook AI Research(FAIR) has become the undisputed champion of self-supervised learning techniques. Recently, FAIR has published self-supervised learning models in language and computer vision that represent major advancements in the space.
OpenAI — Transformers: GPT-3 has made transformer models part of popular culture but OpenAI is not stopping there. Following its breakthroughs in natural language processing, the AI powerhouse has continue applying transformer models into new areas such as computer vision with exciting results.
Google — AutoML: Google was not only one of the pioneers in Automated machine learning(AutoML) but has become one of the biggest producers of researcher and technology in this area. Google is advancing AutoML is highly diverse areas such as time-series analysis or computer vision.
Microsoft — Machine Teaching: Instead of machine learning, Microsoft believes in machine teaching. Microsoft Research has become the number one AI lab in the world pioneering machine teaching research and technology in areas such as speech analysis and computer vision.
Amazon — Transfer Learning: The work in the Alexa digital assistant has made Amazon one of the leading research hubs for transfer learning techniques. From transferring knowledge across different language models to better machine translation techniques, Amazon has been pushing research in the transfer learning space at an incredible pace.
IBM — Quantum Machine Learning: IBM has managed to pioneer technology in many machine learning areas and then loose a leadership position to other tech companies. A recent area in which IBM is pushing the boundaries of research and contributing active is quantum machine learning. At the moment, IBM has the most sophisticated quantum computing stack in the market and is actively using it to evaluate new deep learning techniques.
This list is not exclusive by any stretch but gives you a good idea of the how some of the biggest companies in the world think about the future of deep learning. Companies like Uber, Nvidia or Netflix also deserve to be in this list but their areas of research are more diverse. Interesting times indeed.
Original. Reposted with permission.
- Microsoft Research Trains Neural Networks to Understand What They Read
- How Reading Papers Helps You Be a More Effective Data Scientist
- Baidu Research: 10 Technology Trends in 2021