- Multi-Task Learning – ERNIE 2.0: State-of-the-Art NLP Architecture Intuitively Explained - Oct 2, 2019.
The tech giant Baidu unveiled its state-of-the-art NLP architecture ERNIE 2.0 earlier this year, which scored significantly higher than XLNet and BERT on all tasks in the GLUE benchmark. This major breakthrough in NLP takes advantage of a new innovation called “Continual Incremental Multi-Task Learning”.
AISC, Architecture, Multitask Learning, NLP
- Pre-training, Transformers, and Bi-directionality - Jul 12, 2019.
Bidirectional Encoder Representations from Transformers BERT (Devlin et al., 2018) is a language representation model that combines the power of pre-training with the bi-directionality of the Transformer’s encoder (Vaswani et al., 2017). BERT improves the state-of-the-art performance on a wide array of downstream NLP tasks with minimal additional task-specific training.
AISC, BERT, NLP, Training, Transformer
- Large-Scale Evolution of Image Classifiers - May 16, 2019.
Deep neural networks excel in many difficult tasks, given large amounts of training data and enough processing power. The neural network architecture is an important factor in achieving a highly accurate model... Techniques to automatically discover these neural network architectures are, therefore, very much desirable.
AISC, Evolutionary Algorithm, Image Classification, Image Recognition, Neural Networks
- Interpolation in Autoencoders via an Adversarial Regularizer - Mar 29, 2019.
Adversarially Constrained Autoencoder Interpolation (ACAI; Berthelot et al., 2018) is a regularization procedure that uses an adversarial strategy to create high-quality interpolations of the learned representations in autoencoders.
Adversarial, AISC, Autoencoder, Machine Learning
- GANs Need Some Attention, Too - Mar 5, 2019.
Self-Attention Generative Adversarial Networks (SAGAN; Zhang et al., 2018) are convolutional neural networks that use the self-attention paradigm to capture long-range spatial relationships in existing images to better synthesize new images.
AISC, Attention, Deep Learning, GANs, Image Generation, Machine Learning
