Deep Learning

Forked from terryum’s awesome deep learning papers.


  • Distilling the knowledge in a neural network (2015), G. Hinton et al. [pdf]
  • Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2015), A. Nguyen et al. [pdf]
  • How transferable are features in deep neural networks? (2014), J. Yosinski et al. [pdf]
  • CNN features off-the-Shelf: An astounding baseline for recognition (2014), A. Razavian et al. [pdf]
  • Learning and transferring mid-Level image representations using convolutional neural networks (2014), M. Oquab et al. [pdf]
  • Visualizing and understanding convolutional networks (2014), M. Zeiler and R. Fergus [pdf]
  • Decaf: A deep convolutional activation feature for generic visual recognition (2014), J. Donahue et al. [pdf]

Optimization / Training Techniques

  • Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015), S. Loffe and C. Szegedy [pdf]
  • Delving deep into rectifiers: Surpassing human-level performance on imagenet classification (2015), K. He et al. [pdf]
  • Dropout: A simple way to prevent neural networks from overfitting (2014), N. Srivastava et al. [pdf]
  • Adam: A method for stochastic optimization (2014), D. Kingma and J. Ba [pdf]
  • Improving neural networks by preventing co-adaptation of feature detectors (2012), G. Hinton et al. [pdf]
  • Random search for hyper-parameter optimization (2012) J. Bergstra and Y. Bengio [pdf]

Unsupervised / Generative Models

  • Pixel recurrent neural networks (2016), A. Oord et al. [pdf]
  • Improved techniques for training GANs (2016), T. Salimans et al. [pdf]
  • Unsupervised representation learning with deep convolutional generative adversarial networks (2015), A. Radford et al. [pdf]
  • DRAW: A recurrent neural network for image generation (2015), K. Gregor et al. [pdf]
  • Generative adversarial nets (2014), I. Goodfellow et al. [pdf]
  • Auto-encoding variational Bayes (2013), D. Kingma and M. Welling [pdf]
  • Building high-level features using large scale unsupervised learning (2013), Q. Le et al. [pdf]

Image Segmentation / Object Detection

  • You only look once: Unified, real-time object detection (2016), J. Redmon et al. [pdf]
  • Fully convolutional networks for semantic segmentation (2015), J. Long et al. [pdf]
  • Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015), S. Ren et al. [pdf]
  • Fast R-CNN (2015), R. Girshick [pdf]
  • Rich feature hierarchies for accurate object detection and semantic segmentation (2014), R. Girshick et al. [pdf]
  • Semantic image segmentation with deep convolutional nets and fully connected CRFs, L. Chen et al. [pdf]
  • Learning hierarchical features for scene labeling (2013), C. Farabet et al. [pdf]

Image / Video

  • Image Super-Resolution Using Deep Convolutional Networks (2016), C. Dong et al. [pdf]
  • A neural algorithm of artistic style (2015), L. Gatys et al. [pdf]
  • Deep visual-semantic alignments for generating image descriptions (2015), A. Karpathy and L. Fei-Fei [pdf]
  • Show, attend and tell: Neural image caption generation with visual attention (2015), K. Xu et al. [pdf]
  • Show and tell: A neural image caption generator (2015), O. Vinyals et al. [pdf]
  • Long-term recurrent convolutional networks for visual recognition and description (2015), J. Donahue et al. [pdf]
  • VQA: Visual question answering (2015), S. Antol et al. [pdf]
  • DeepFace: Closing the gap to human-level performance in face verification (2014), Y. Taigman et al. [pdf]:
  • Large-scale video classification with convolutional neural networks (2014), A. Karpathy et al. [pdf]
  • DeepPose: Human pose estimation via deep neural networks (2014), A. Toshev and C. Szegedy [pdf]
  • Two-stream convolutional networks for action recognition in videos (2014), K. Simonyan et al. [pdf]
  • 3D convolutional neural networks for human action recognition (2013), S. Ji et al. [pdf]

Natural Language Processing

  • Neural Architectures for Named Entity Recognition (2016), G. Lample et al. [pdf]
  • Exploring the limits of language modeling (2016), R. Jozefowicz et al. [pdf]
  • Teaching machines to read and comprehend (2015), K. Hermann et al. [pdf]
  • Effective approaches to attention-based neural machine translation (2015), M. Luong et al. [pdf]
  • Conditional random fields as recurrent neural networks (2015), S. Zheng and S. Jayasumana. [pdf]
  • Memory networks (2014), J. Weston et al. [pdf]
  • Neural turing machines (2014), A. Graves et al. [pdf]
  • Neural machine translation by jointly learning to align and translate (2014), D. Bahdanau et al. [pdf]
  • Sequence to sequence learning with neural networks (2014), I. Sutskever et al. [pdf]
  • Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014), K. Cho et al. [pdf]
  • A convolutional neural network for modeling sentences (2014), N. Kalchbrenner et al. [pdf]
  • Convolutional neural networks for sentence classification (2014), Y. Kim [pdf]
  • Glove: Global vectors for word representation (2014), J. Pennington et al. [pdf]
  • Distributed representations of sentences and documents (2014), Q. Le and T. Mikolov [pdf]
  • Distributed representations of words and phrases and their compositionality (2013), T. Mikolov et al. [pdf]
  • Efficient estimation of word representations in vector space (2013), T. Mikolov et al. [pdf]
  • Recursive deep models for semantic compositionality over a sentiment treebank (2013), R. Socher et al. [pdf]
  • Generating sequences with recurrent neural networks (2013), A. Graves. [pdf]

Speech / Other

  • End-to-end attention-based large vocabulary speech recognition (2016), D. Bahdanau et al. [pdf]
  • Deep speech 2: End-to-end speech recognition in English and Mandarin (2015), D. Amodei et al. [pdf]
  • Speech recognition with deep recurrent neural networks (2013), A. Graves [pdf]
  • Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups (2012), G. Hinton et al. [pdf]
  • Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition (2012) G. Dahl et al. [pdf]
  • Acoustic modeling using deep belief networks (2012), A. Mohamed et al. [pdf]

Reinforcement Learning

  • End-to-end training of deep visuomotor policies (2016), S. Levine et al. [pdf]
  • Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection (2016), S. Levine et al. [pdf]
  • Asynchronous methods for deep reinforcement learning (2016), V. Mnih et al. [pdf]
  • Deep Reinforcement Learning with Double Q-Learning (2016), H. Hasselt et al. [pdf]
  • Mastering the game of Go with deep neural networks and tree search (2016), D. Silver et al. [pdf]
  • Continuous control with deep reinforcement learning (2015), T. Lillicrap et al. [pdf]
  • Human-level control through deep reinforcement learning (2015), V. Mnih et al. [pdf]
  • Deep learning for detecting robotic grasps (2015), I. Lenz et al. [pdf]
  • Playing atari with deep reinforcement learning (2013), V. Mnih et al. [pdf]

New papers

  • Deep Photo Style Transfer (2017), F. Luan et al. [pdf]
  • Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017), T. Salimans et al. [pdf]
  • Deformable Convolutional Networks (2017), J. Dai et al. [pdf]
  • Mask R-CNN (2017), K. He et al. [pdf]
  • Learning to discover cross-domain relations with generative adversarial networks (2017), T. Kim et al. [pdf]
  • Deep voice: Real-time neural text-to-speech (2017), S. Arik et al., [pdf]
  • PixelNet: Representation of the pixels, by the pixels, and for the pixels (2017), A. Bansal et al. [pdf]
  • Batch renormalization: Towards reducing minibatch dependence in batch-normalized models (2017), S. Ioffe. [pdf]
  • Wasserstein GAN (2017), M. Arjovsky et al. [pdf]
  • Understanding deep learning requires rethinking generalization (2017), C. Zhang et al. [pdf]
  • Least squares generative adversarial networks (2016), X. Mao et al. [pdf]

Classic Papers

  • An analysis of single-layer networks in unsupervised feature learning (2011), A. Coates et al. [pdf]
  • Deep sparse rectifier neural networks (2011), X. Glorot et al. [pdf]
  • Natural language processing (almost) from scratch (2011), R. Collobert et al. [pdf]
  • Recurrent neural network based language model (2010), T. Mikolov et al. [pdf]
  • Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion (2010), P. Vincent et al. [pdf]
  • Learning mid-level features for recognition (2010), Y. Boureau [pdf]
  • A practical guide to training restricted boltzmann machines (2010), G. Hinton [pdf]
  • Understanding the difficulty of training deep feedforward neural networks (2010), X. Glorot and Y. Bengio [pdf]
  • Why does unsupervised pre-training help deep learning (2010), D. Erhan et al. [pdf]
  • Learning deep architectures for AI (2009), Y. Bengio. [pdf]
  • Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations (2009), H. Lee et al. [pdf]
  • Greedy layer-wise training of deep networks (2007), Y. Bengio et al. [pdf]
  • A fast learning algorithm for deep belief nets (2006), G. Hinton et al. [pdf]
  • Gradient-based learning applied to document recognition (1998), Y. LeCun et al. [pdf]
  • Long short-term memory (1997), S. Hochreiter and J. Schmidhuber. [pdf]