Deep Learning Scaling is Predictable, Empirically

1 December 2017

Sharan Narang

Md. Mostofa Ali Patwary

Yang Yang

Yanqi Zhou

ArXiv PDF HTML

Papers citing "Deep Learning Scaling is Predictable, Empirically"

36 / 386 papers shown

Title
Data Valuation using Reinforcement Learning Jinsung Yoon Sercan Ö. Arik Tomas Pfister TDI 27 173 0 25 Sep 2019
Beyond Human-Level Accuracy: Computational Challenges in Deep Learning Joel Hestness Newsha Ardalani G. Diamos 13 66 0 03 Sep 2019
Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning Linchao Zhu Sercan Ö. Arik Yezhou Yang Tomas Pfister 25 5 0 29 Aug 2019
P2L: Predicting Transfer Learning for Images and Semantic Relations Bishwaranjan Bhattacharjee J. Kender Matthew Q. Hill Parijat Dube Siyu Huo Michael R. Glass Brian M. Belgodere Sharath Pankanti Noel Codella Patrick Watson VLM 16 12 0 20 Aug 2019
TabNet: Attentive Interpretable Tabular Learning Sercan Ö. Arik Tomas Pfister LMTD 55 1,283 0 20 Aug 2019
Human uncertainty makes classification more robust Joshua C. Peterson Ruairidh M. Battleday Thomas Griffiths Olga Russakovsky OOD 6 294 0 19 Aug 2019
Less (Data) Is More: Why Small Data Holds the Key to the Future of Artificial Intelligence C. Greco A. Polonioli Jacopo Tagliabue 21 4 0 22 Jul 2019
Minimizers of the Empirical Risk and Risk Monotonicity Marco Loog T. Viering A. Mey 14 26 0 11 Jul 2019
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges N. Arivazhagan Ankur Bapna Orhan Firat Dmitry Lepikhin Melvin Johnson ... George F. Foster Colin Cherry Wolfgang Macherey Z. Chen Yonghui Wu 34 423 0 11 Jul 2019
Selection via Proxy: Efficient Data Selection for Deep Learning Cody Coleman Christopher Yeh Stephen Mussmann Baharan Mirzasoleiman Peter Bailis Percy Liang J. Leskovec Matei A. Zaharia 26 329 0 26 Jun 2019
One Epoch Is All You Need Aran Komatsuzaki 11 50 0 16 Jun 2019
Landslide Geohazard Assessment With Convolutional Neural Networks Using Sentinel-2 Imagery Data Silvia L. Ullo Maximillian S. Langenkamp Tuomas P. Oikarinen M. P. D. Rosso A. Sebastianelli Federica Piccirillo S. Sica 6 25 0 10 Jun 2019
Multimodal End-to-End Autonomous Driving Yi Xiao Felipe Codevilla A. Gurram O. Urfalioglu Antonio M. López 19 240 0 07 Jun 2019
Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm S. Spigler Mario Geiger M. Wyart 22 38 0 26 May 2019
A Survey on Face Data Augmentation Xiang Wang Kai Wang Kai Wang CVBM 20 130 0 26 Apr 2019
Deep Learning in steganography and steganalysis from 2015 to 2018 Marc Chaumont 11 48 0 31 Mar 2019
Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools R. Mayer Hans-Arno Jacobsen GNN 27 186 0 27 Mar 2019
A Brain-inspired Algorithm for Training Highly Sparse Neural Networks Zahra Atashgahi Joost Pieterse Shiwei Liu Decebal Constantin Mocanu Raymond N. J. Veldhuis Mykola Pechenizkiy 35 15 0 17 Mar 2019
The State of Sparsity in Deep Neural Networks Trevor Gale Erich Elsen Sara Hooker 21 743 0 25 Feb 2019
Combining learning rate decay and weight decay with complexity gradient descent - Part I Pierre Harvey Richemond Yike Guo 25 4 0 07 Feb 2019
Technical Considerations for Semantic Segmentation in MRI using Convolutional Neural Networks Arjun D Desai G. Gold B. Hargreaves Akshay S. Chaudhari 19 30 0 05 Feb 2019
Impact of Training Dataset Size on Neural Answer Selection Models Trond Linjordet K. Balog 17 38 0 29 Jan 2019
Machine learning in resting-state fMRI analysis Meenakshi Khosla K. Jamison G. Ngo Amy Kuceyeski M. Sabuncu 16 168 0 30 Dec 2018
An Empirical Model of Large-Batch Training Sam McCandlish Jared Kaplan Dario Amodei OpenAI Dota Team 15 268 0 14 Dec 2018
On the potential for open-endedness in neural networks N. Guttenberg N. Virgo A. Penn 21 10 0 12 Dec 2018
Extractive Summary as Discrete Latent Variables Aran Komatsuzaki 15 3 0 14 Nov 2018
Measuring the Effects of Data Parallelism on Neural Network Training Christopher J. Shallue Jaehoon Lee J. Antognini J. Mamou J. Ketterling Yao Wang 49 407 0 08 Nov 2018
Language Modeling at Scale Md. Mostofa Ali Patwary Milind Chabbi Heewoo Jun Jiaji Huang G. Diamos Kenneth Church ALM 28 5 0 23 Oct 2018
Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis Kelly W. Zhang Samuel R. Bowman 32 70 0 26 Sep 2018
Large Scale Language Modeling: Converging on 40GB of Text in Four Hours Raul Puri Robert M. Kirby Nikolai Yakovenko Bryan Catanzaro 19 29 0 03 Aug 2018
Decreasing the size of the Restricted Boltzmann machine Yohei Saito Takuya Kato 11 3 0 09 Jul 2018
Why Is My Classifier Discriminatory? Irene Y. Chen Fredrik D. Johansson David Sontag FaML 6 393 0 30 May 2018
Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation Zhong Zhou Matthias Sperber A. Waibel 8 12 0 21 Apr 2018
Challenging Images For Minds and Machines Amir Rosenfeld John K. Tsotsos VLM 23 1 0 13 Feb 2018
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 273 1,896 0 10 Jan 2017
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 218 7,926 0 17 Aug 2015