Awesome Deep Learning: Most Cited Deep Learning Papers
Awesome Deep Learning: Most Cited Deep Learning PapersThis post introduces a curated list of the most cited deep learning papers (since 2012), provides the inclusion criteria, shares a few entry examples, and points to the full listing for those interested in investigating further.
By Terry Taewoong Um, University of Waterloo.
We believe that there exist classic deep learning papers which are worth reading regardless of their application domain. Rather than providing overwhelming amount of papers, We would like to provide a curated list of the awesome deep learning papers which are considered as must-reads in certain research domains.
Background
Before this list, there exist other awesome deep learning lists, for example, Deep Vision and Awesome Recurrent Neural Networks. Also, after this list comes out, another awesome list for deep learning beginners, called Deep Learning Papers Reading Roadmap, has been created and loved by many deep learning researchers.
Although the Roadmap List includes lots of important deep learning papers, it feels overwhelming for me to read them all. As I mentioned in the introduction, I believe that seminal works can give us lessons regardless of their application domain. Thus, I would like to introduce top 100 deep learning papers here as a good starting point of overviewing deep learning researches.
To get the news for newly released papers everyday, follow my twitter or facebook page!
- A list of top 100 deep learning papers published from 2012 to 2016 is suggested.
- If a paper is added to the list, another paper (usually from *More Papers from 2016" section) should be removed to keep top 100 papers. (Thus, removing papers is also important contributions as well as adding papers)
- Papers that are important, but failed to be included in the list, will be listed in More than Top 100 section.
- Please refer to New Papers and Old Papers sections for the papers published in recent 6 months or before 2012.
Citation criteria
- < 6 months : New Papers (by discussion)
- 2016 : +60 citations or "More Papers from 2016"
- 2015 : +200 citations
- 2014 : +400 citations
- 2013 : +600 citations
- 2012 : +800 citations
- ~2012 : Old Papers (by discussion)
Please note that we prefer seminal deep learning papers that can be applied to various researches rather than application papers. For that reason, some papers that meet the criteria may not be accepted while others can be. It depends on the impact of the paper, applicability to other researches scarcity of the research domain, and so on.
Editor: What follows is a selection of curated papers from this list, one from each category, as selected by the author. Please see the original for the full listing.
1. Understanding / Generalization / Transfer
Distilling the knowledge in a neural network (2015), G. Hinton et al. [pdf]
2. Optimization / Training Techniques
Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015), S. Loffe and C. Szegedy [pdf]
3. Unsupervised / Generative Models
Unsupervised representation learning with deep convolutional generative adversarial networks (2015), A. Radford et al. [pdf]
4. Convolutional Neural Network Models
Deep residual learning for image recognition (2016), K. He et al. [pdf]
5. Image: Segmentation / Object Detection
Fast R-CNN (2015), R. Girshick [pdf]
6. Image / Video / Etc.
Show and tell: A neural image caption generator (2015), O. Vinyals et al. [pdf]
7. Natural Language Processing / RNNs
Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014), K. Cho et al. [pdf]
8. Speech / Other Domain
Speech recognition with deep recurrent neural networks (2013), A. Graves [pdf]
9. Reinforcement Learning / Robotics
Human-level control through deep reinforcement learning (2015), V. Mnih et al. [pdf]
10. More Papers from 2016
Domain-adversarial training of neural networks (2016), Y. Ganin et al. [pdf]
Bio: Terry Taewoong Um is a PhD candidate at U. Waterloo. Terry completed his B.S and M.S. in Mechanical and Aerospace Engineering at the Seoul National University in 2008, 2010, respectively. He also worked at LIG Nex1 and Korea Institute of Science and Technology (KIST) until 2014. His previous research was on robot motion planning and power-assist exoskeleton. In his Ph.D. study, he is focusing on introducing Lie group geometry to deep learning techniques for learning human / robot motions.
Original. Reposted with permission.
Related: