-
GPT-3, a giant step for Deep Learning and NLP?
Recently, OpenAI announced a new successor to their language model, GPT-3, that is now the largest model trained so far with 175 billion parameters. Training a language model this large has its merits and limitations, so this article covers some of its most interesting and important aspects.
-
Preparing for the Unexpected
In some domains, new values appear all the time, so it's crucial to handle them in a good way. Using deep learning, one can learn a special Out-of-Vocabulary embedding for these new values. But how can you train this embedding to generalize well to any unseen value? We explain one of the methods employed at Taboola.
-
Variational Autoencoders Explained in Detail
We explain how to implement VAE - including simple to understand tensorflow code using MNIST and a cool trick of how you can generate an image of a digit conditioned on the digit.
-
How to Engineer Your Way Out of Slow Models
We describe how we handle performance issues with our deep learning models, including how to find subgraphs that take a lot of calculation time and how to extract these into a caching mechanism.
-
Word Morphing – an original idea
In this post, we describe how to utilise word2vec's embeddings and A* search algorithm to morph between words.
-
Using Uncertainty to Interpret your Model
We outline why you should care about uncertainty and discuss the different types, including model, data and measurement uncertainty and what different purposes these all serve.
-
Linear Regression in the Wild
We take a look at how to use linear regression when the dependent variables have measurement errors.
|
Algorithm Engineer at AI21 Labs, in the mission of building AI systems with the capacity to understand and generate natural language.
