- Deep Compression: Optimization Techniques for Inference & Efficiency - Mar 20, 2019.
We explain deep compression for improved inference efficiency, mobile applications, and regularization as technology cozies up to the physical limits of Moore's law.
Compression, Convolutional Neural Networks, Deep Learning, ICLR, Inference, Optimization, Regularization
- 10 Exciting Ideas of 2018 in NLP - Jan 16, 2019.
We outline a selection of exciting developments in NLP from the last year, and include useful recent papers and images to help further assist with your learning.
BERT, Bias, ICLR, Machine Translation, NLP, Transformer, Unsupervised Learning
- The Two Phases of Gradient Descent in Deep Learning - May 12, 2017.
In short, you reach different resting placing with different SGD algorithms. That is, different SGDs just give you differing convergence rates due to different strategies, but we do expect that they all end up at the same results!
Deep Learning, ICLR, Neural Networks
- The ICLR Experiment: Deep Learning Pioneers Take on Scientific Publishing - Feb 15, 2016.
Deep learning pioneers Yann LeCun and Yoshua Bengio have undertaken a grand experiment in academic publishing. Embracing a radical level of transparency and unprecedented public participation, they've created an opportunity not only to find and vet the best papers, but also to gather data about the publication process itself.
Academics, arXiv, Deep Learning, ICLR, Neural Networks, Yann LeCun, Yoshua Bengio, Zachary Lipton