v1v2 (latest)

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

8 June 2017

Piotr Dollár

Papers citing "Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour"

50 / 2,054 papers shown

Title
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning Sai Praneeth Karimireddy Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 77 348 0 14 Oct 2019
On Empirical Comparisons of Optimizers for Deep Learning Dami Choi Christopher J. Shallue Zachary Nado Jaehoon Lee Chris J. Maddison George E. Dahl 129 259 0 11 Oct 2019
Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey Bin Qian Jie Su Z. Wen D. N. Jha Yinhao Li ... Albert Y. Zomaya Omer F. Rana Lizhe Wang Maciej Koutny R. Ranjan 71 4 0 11 Oct 2019
Rosetta: Large scale system for text detection and recognition in images Fedor Borisyuk Albert Gordo V. Sivakumar 92 300 0 11 Oct 2019
Blink: Fast and Generic Collectives for Distributed ML Guanhua Wang Shivaram Venkataraman Amar Phanishayee J. Thelin Nikhil R. Devanur Ion Stoica VLM 65 142 0 11 Oct 2019
On the adequacy of untuned warmup for adaptive optimization Jerry Ma Denis Yarats 106 70 0 09 Oct 2019
Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation Hang Gao Xizhou Zhu Steve Lin Jifeng Dai 130 65 0 07 Oct 2019
Parallelizing Training of Deep Generative Models on Massive Scientific Datasets S. A. Jacobs B. Van Essen D. Hysom Jae-Seung Yeom Tim Moon ... J. Gaffney Tom Benson Peter B. Robinson L. Peterson B. Spears BDL AI4CE 77 17 0 05 Oct 2019
Distributed Learning of Deep Neural Networks using Independent Subnet Training John Shelton Hyatt Cameron R. Wolfe Michael Lee Yuxin Tang Anastasios Kyrillidis Christopher M. Jermaine OOD 92 39 0 04 Oct 2019
Farkas layers: don't shift the data, fix the geometry Aram-Alexandre Pooladian Chris Finlay Adam M. Oberman AI4CE 36 1 0 04 Oct 2019
SELF: Learning to Filter Noisy Labels with Self-Ensembling Philipp Kratzer Marc Toussaint Thi Phuong Nhung Ngo T. Nguyen Jim Mainprice Thomas Brox NoLa 96 317 0 04 Oct 2019
SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead A. Masullo Ligang He Toby Perrett Rui Mao Carsten Maple Majid Mirmehdi 106 318 0 03 Oct 2019
Accelerating Data Loading in Deep Neural Network Training Chih-Chieh Yang Guojing Cong 78 38 0 02 Oct 2019
MLPerf Training Benchmark Arya D. McCarthy Christine Cheng Cody Coleman Greg Diamos Paulius Micikevicius ... Carole-Jean Wu Lingjie Xu Masafumi Yamazaki C. Young Matei A. Zaharia 124 316 0 02 Oct 2019
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum Jianyu Wang Vinayak Tantia Nicolas Ballas Michael G. Rabbat 99 201 0 01 Oct 2019
Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videos Ji Lin Chuang Gan Song Han 78 10 0 01 Oct 2019
The Non-IID Data Quagmire of Decentralized Machine Learning Kevin Hsieh Amar Phanishayee O. Mutlu Phillip B. Gibbons 192 576 0 01 Oct 2019
SURREAL-System: Fully-Integrated Stack for Distributed Deep Reinforcement Learning Linxi Fan Yuke Zhu Jiren Zhu Zihua Liu Orien Zeng Anchit Gupta Joan Creus-Costa Silvio Savarese Li Fei-Fei OffRL GNN 89 3 0 27 Sep 2019
At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks? Niv Giladi Mor Shpigel Nacson Elad Hoffer Daniel Soudry 80 22 0 26 Sep 2019
Elastic deep learning in multi-tenant GPU cluster Yidi Wu Kaihao Ma Xiao Yan Zhi Liu Zhenkun Cai Yuzhen Huang James Cheng Han Yuan Fan Yu 25 2 0 26 Sep 2019
Revisiting Knowledge Distillation via Label Smoothing Regularization Li-xin Yuan Francis E. H. Tay Guilin Li Tao Wang Jiashi Feng 73 91 0 25 Sep 2019
Speech Recognition with Augmented Synthesized Speech Andrew Rosenberg Yu Zhang Bhuvana Ramabhadran Ye Jia Pedro J. Moreno Yonghui Wu Zelin Wu 69 128 0 25 Sep 2019
Gap Aware Mitigation of Gradient Staleness Saar Barkai Ido Hakimi Assaf Schuster 89 23 0 24 Sep 2019
Machine Learning Pipelines with Modern Big Data Tools for High Energy Physics M. Migliorini R. Castellotti L. Canali M. Zanetti 64 7 0 23 Sep 2019
Scale MLPerf-0.6 models on Google TPU-v3 Pods Sameer Kumar Victor Bitorff Dehao Chen Chi-Heng Chou Blake A. Hechtman ... Peter Mattson Shibo Wang Tao Wang Yuanzhong Xu Zongwei Zhou 89 39 0 21 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 365 1,926 0 17 Sep 2019
Heterogeneity-Aware Asynchronous Decentralized Training Qinyi Luo Jiaao He Youwei Zhuo Xuehai Qian 62 8 0 17 Sep 2019
Ouroboros: On Accelerating Training of Transformer-Based Language Models Qian Yang Zhouyuan Huo Wenlin Wang Heng-Chiao Huang Lawrence Carin 57 9 0 14 Sep 2019
DL2: A Deep Learning-driven Scheduler for Deep Learning Clusters Size Zheng Yixin Bao Yangrui Chen Chuan Wu Chen Meng Wei Lin 56 85 0 13 Sep 2019
DARTS+: Improved Differentiable Architecture Search with Early Stopping Hanwen Liang Shifeng Zhang Jiacheng Sun Xingqiu He Weiran Huang Kechen Zhuang Zhenguo Li 100 290 0 13 Sep 2019
CARS: Continuous Evolution for Efficient Neural Architecture Search Zhaohui Yang Yunhe Wang Xinghao Chen Boxin Shi Chao Xu Chunjing Xu Qi Tian Chang Xu 122 231 0 11 Sep 2019
Addressing Algorithmic Bottlenecks in Elastic Machine Learning with Chicle Michael Kaufmann K. Kourtis Celestine Mendler-Dünner Adrian Schüpbach Thomas Parnell 15 0 0 11 Sep 2019
Towards Understanding the Importance of Shortcut Connections in Residual Networks Tianyi Liu Minshuo Chen Mo Zhou S. Du Enlu Zhou T. Zhao 49 45 0 10 Sep 2019
Distributed Equivalent Substitution Training for Large-Scale Recommender Systems Haidong Rong Yangzihao Wang Feihu Zhou Junjie Zhai Haiyang Wu ... Fan Li Han Zhang Yuekui Yang Zhenyu Guo Di Wang OffRL 51 11 0 10 Sep 2019
Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum Yosuke Shinya E. Simo-Serra Taiji Suzuki 48 12 0 09 Sep 2019
Distributed Training of Embeddings using Graph Analytics G. Gill Roshan Dathathri Saeed Maleki Madan Musuvathi Todd Mytkowicz Olli Saarikivi The University of Texas at Austin GNN 26 1 0 08 Sep 2019
Linear Context Transform Block D. Ruan Jun Wen Nenggan Zheng Min Zheng ViT 32 23 0 06 Sep 2019
Minibatch Processing in Spiking Neural Networks D. J. Saunders Cooper Sigrist Kenneth Chaney R. Kozma H. Siegelmann 44 3 0 05 Sep 2019
Hierarchical Federated Learning Across Heterogeneous Cellular Networks Mehdi Salehi Heydar Abad Emre Ozfatura Deniz Gunduz Ozgur Ercetin FedML 143 314 0 05 Sep 2019
POD: Practical Object Detection with Scale-Sensitive Network Junran Peng Ming Sun Zhaoxiang Zhang Tieniu Tan Junjie Yan ObjD 88 22 0 05 Sep 2019
Beyond Human-Level Accuracy: Computational Challenges in Deep Learning Joel Hestness Newsha Ardalani G. Diamos 64 68 0 03 Sep 2019
Training-Time-Friendly Network for Real-Time Object Detection Zili Liu Tu Zheng Guodong Xu Zheng Yang Haifeng Liu Deng Cai ObjD TTA 83 87 0 02 Sep 2019
TapirXLA: Embedding Fork-Join Parallelism into the XLA Compiler in TensorFlow Using Tapir S. Samsi Michael Houle 24 4 0 29 Aug 2019
Distributed Deep Learning for Precipitation Nowcasting S. Samsi Christopher J. Mattioli Mark S. Veillette 77 23 0 28 Aug 2019
Push for Center Learning via Orthogonalization and Subspace Masking for Person Re-Identification Weinong Wang Wenjie Pei Qiong Cao Shu Liu Yu-Wing Tai 24 1 0 28 Aug 2019
Unsupervised Deep Feature Transfer for Low Resolution Image Classification Yuanwei Wu Ziming Zhang Guanghui Wang 71 22 0 27 Aug 2019
Curved Text Detection in Natural Scene Images with Semi- and Weakly-Supervised Learning Xugong Qin Yu Zhou Dongbao Yang Weiping Wang 57 27 0 27 Aug 2019
SeesawFaceNets: sparse and robust face verification model for mobile platform Jintao Zhang 3DH CVBM 55 9 0 24 Aug 2019
Dynamic Scheduling of MPI-based Distributed Deep Learning Training Jobs Tim Capes Vishal Raheja Mete Kemertas Iqbal Mohomed AI4CE 25 3 0 21 Aug 2019
Instance Scale Normalization for image understanding Zewen He He Huang Yudong Wu Guan Huang Wensheng Zhang ObjD 23 0 0 20 Aug 2019