10 Amazing Machine Learning Projects of 2020
So much progress in AI and machine learning happened in 2020, especially in the areas of AI-generating creativity and low-to-no-code frameworks. Check out these trending and popular machine learning projects released last year, and let them inspire your work throughout 2021.
By Anupam Chugh, iOS Developer | Writer @Medium | BITS Pilani.
Photo by Paul Hanaoka on Unsplash.
A lot happened in the machine learning community during the past year. Here’s a tour through the most popular and trending open-source research projects, demos, and prototypes. It ranges from photo editing to NLP to training models with “no-code,” and I hope they inspire you to build incredible AI-powered products this year. You can also find additional machine learning projects here.
1. Background Matting v2
Background Matting v2 takes a cue from the popular The World is Your Green Screen open-source project and showcases how to remove or change the background in real-time. It gives better performance (30fps at 4K and 60fps at FHD) and can be used with Zoom, the popular videoconferencing app.
The technique uses an additional captured frame of the background and uses it in recovering the alpha matte and the foreground layer. To process high-resolution images in real-time, two neural networks are used.
This project is certainly helpful if you’re looking to remove a person from a video whilst preserving the background.
2. SkyAR
Here’s another amazing project that does a video sky replacement and harmonization, which can automatically generate realistic and dramatic sky backgrounds in videos with controllable styles.
Based on Pytorch, the project partially adapts code from the pytorch-CycleGAN-and-pix2pix project, and it leverages sky matting, motion estimation through optical flow, and image blending to provide artistic backgrounds on videos in real-time.
The above open-source has incredible potential in movies and video games such as adding fake rain/sunny weather/etc.
3. AnimeGAN v2
Cartoonifying photos is always a fun machine learning project. Isn’t it?
This project, AnimeGANv2, is the improved version of AnimeGAN. Specifically, it combines neural style transfer and generative adversarial network (GAN) to accomplish the task while also ensuring that the generation of high-frequency artifacts is prevented.
4. txtai
AI-refined search engines and QA Chatbots are always the need of the hour. And that’s exactly what this project does.
By using sentence-transformers, transformers, and faiss, txtai builds an AI-powered engine for contextual search and extractive question-answering.
Essentially, txtai supports building text indices to perform similarity searches and create extractive question-answering based systems.
5. Bringing-Old-Photos-Back-to-Life
Next up, we have Microsoft’s latest photo restoration project that auto-fixes damaged photos.
Specifically, it restores old photos that suffer from complex degradation through a deep learning implementation in PyTorch by leveraging scratch detection, face enhancements, and other techniques.
As per their research paper: “We train two variational autoencoders (VAEs) to respectively transform old photos and clean photos into two latent spaces. And the translation between these two latent spaces is learned with synthetic paired data. This translation generalizes well to real photos because the domain gap is closed in the compact latent space. Besides, to address multiple degradations mixed in one old photo, we design a global branch with a partial nonlocal block targeting the structured defects, such as scratches and dust spots, and a local branch targeting the unstructured defects, such as noises and blurriness.”
The model certainly outperforms the conventional state of the art methods, as evident in the below demo:
6. Avatarify
Deepfake projects have taken the machine learning and AI community by storm. This project shows a classic example of that by letting you create photo-realistic avatars in real-time video conferencing apps.
Basically, it uses the First Order Model to do a motion extraction from the video and apply it to the target avatar image by using optical flow. In doing so, you can generate avatars on a virtual camera or even animate classic paintings. From Elon Musk to Mona Lisa, you can impersonate anyone for fun!
7. Pulse
Here’s an AI model that showcases how to generate a realistic face image from a low-resolution one.
PULSE, which stands for Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models, provides an alternative formulation of the super-resolution problem based on creating realistic SR images that also downscale back correctly.
8. pixel2style2pixel
Based on the research paper “Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation,” this project uses a Pixel2Pixel framework, and it’s designed to address a wide range of image-to-image tasks using the same architecture in order to avoid any possible locality bias.
Based on a novel encoder network, this network can be trained to align a face image to a frontal pose, conditional image synthesis, and create super-resolution images.
From generating almost real-life people out of cartoonist pics to converting sketches or face segmentations to photo-realistic images, there are so many things that you can do with this.
9. igel
It could be due to budget issues or a lack of a clear vision, but finding people with relevant machine learning expertise is always a challenge for startups. More so since the field is always a work in progress.
Hence, there has been a surge in no-code machine learning platforms lately, with the likes of Google and Apple releasing their own toolsets to quickly train models.
This delightful open-source machine learning project does just that by allowing you to train/fit, test, and use models without writing code. While the GUI drag and drop version is still a work in progress, there’s a lot that you can achieve with the command line tools of this project:
//train or fit a model igel fit -dp 'path_to_your_csv_dataset.csv' -yml 'path_to_your_yaml_file.yaml' //evaluate igel evaluate -dp 'path_to_your_evaluation_dataset.csv' //predict igel predict -dp 'path_to_your_test_dataset.csv'
There’s also a single command igel experiment to combine all the phases: train, evaluate, and predict. For more details, refer to the documentation here.
10. Pose Animator
Last but not least, we have a web animation tool. Basically, this project uses PoseNet and FaceMesh landmark results to bring an SVG vector image to life by leveraging some TensorFlow.js models.
You can animate your own designs or skeleton images in the following way:
Original. Reposted with permission.
Related: