v1v2 (latest)

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

8 June 2017

Piotr Dollár

Papers citing "Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour"

50 / 2,054 papers shown

Title
Hoplite: Efficient and Fault-Tolerant Collective Communication for Task-Based Distributed Systems Siyuan Zhuang Zhuohan Li Danyang Zhuo Stephanie Wang Eric Liang Robert Nishihara Philipp Moritz Ion Stoica 40 24 0 13 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 442 18,989 0 13 Feb 2020
Scalable and Practical Natural Gradient for Large-Scale Deep Learning Kazuki Osawa Yohei Tsuji Yuichiro Ueno Akira Naruse Chuan-Sheng Foo Rio Yokota 85 37 0 13 Feb 2020
The Conditional Entropy Bottleneck Ian S. Fischer OOD 125 122 0 13 Feb 2020
On Layer Normalization in the Transformer Architecture Ruibin Xiong Yunchang Yang Di He Kai Zheng Shuxin Zheng Chen Xing Huishuai Zhang Yanyan Lan Liwei Wang Tie-Yan Liu AI4CE 160 1,005 0 12 Feb 2020
BatchLayout: A Batch-Parallel Force-Directed Graph Layout Algorithm in Shared Memory Md. Khaledur Rahman Majedul Haque Sujon A. Azad 30 11 0 11 Feb 2020
Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts Max Ryabinin Anton I. Gusev FedML 91 52 0 10 Feb 2020
Momentum Improves Normalized SGD Ashok Cutkosky Harsh Mehta ODL 107 128 0 09 Feb 2020
How to train your neural ODE: the world of Jacobian and kinetic regularization Chris Finlay J. Jacobsen L. Nurbekyan Adam M. Oberman 94 302 0 07 Feb 2020
Automatic image-based identification and biomass estimation of invertebrates J. Ärje C. Melvad M. R. Jeppesen S. A. Madsen Jenni Raitoharju ... Alexandros Iosifidis V. Tirronen Kristian Meissner Moncef Gabbouj T. Høye 42 73 0 05 Feb 2020
Large Batch Training Does Not Need Warmup Zhouyuan Huo Bin Gu Heng-Chiao Huang AI4CE ODL 52 5 0 04 Feb 2020
Improving Efficiency in Large-Scale Decentralized Distributed Training Wei Zhang Xiaodong Cui Abdullah Kayi Mingrui Liu Ulrich Finkler ... Youssef Mroueh A. Buyuktosunoglu Payel Das David S. Kung M. Picheny 52 16 0 04 Feb 2020
Radioactive data: tracing through training Alexandre Sablayrolles Matthijs Douze Cordelia Schmid Hervé Jégou 102 76 0 03 Feb 2020
Dynamic Parameter Allocation in Parameter Servers Alexander Renz-Wieland Rainer Gemulla Steffen Zeuch Volker Markl 52 18 0 03 Feb 2020
SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks Sungho Shin Yoonho Boo Wonyong Sung MQ 44 3 0 02 Feb 2020
A Proof of Useful Work for Artificial Intelligence on the Blockchain Andrei Lihu Jincheng Du Igor Barjaktarevic Patrick Gerzanics Mark Harvilla 63 32 0 25 Jan 2020
Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation Byung Hoon Ahn Prannoy Pilligundla Amir Yazdanbakhsh H. Esmaeilzadeh ODL 113 82 0 23 Jan 2020
Audiovisual SlowFast Networks for Video Recognition Fanyi Xiao Yong Jae Lee Kristen Grauman Jitendra Malik Christoph Feichtenhofer 280 209 0 23 Jan 2020
Intermittent Pulling with Local Compensation for Communication-Efficient Federated Learning Yining Qi Zhihao Qu Song Guo Xin Gao Ruixuan Li Baoliu Ye FedML 45 9 0 22 Jan 2020
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence Kihyuk Sohn David Berthelot Chun-Liang Li Zizhao Zhang Nicholas Carlini E. D. Cubuk Alexey Kurakin Han Zhang Colin Raffel AAML 169 3,602 0 21 Jan 2020
Joint Learning of Instance and Semantic Segmentation for Robotic Pick-and-Place with Heavy Occlusions in Clutter Kentaro Wada K. Okada Masayuki Inaba VLM ISeg 54 27 0 21 Jan 2020
Instance Segmentation of Visible and Occluded Regions for Finding and Picking Target from a Pile of Objects Kentaro Wada Shingo Kitagawa K. Okada Masayuki Inaba ISeg SSeg 93 24 0 21 Jan 2020
Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard Zoltán Tüske G. Saon Kartik Audhkhasi Brian Kingsbury BDL 101 69 0 20 Jan 2020
Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network Jungkyu Lee Taeryun Won Tae Kwan Lee Hyemin Lee Geonmo Gu K. Hong 97 57 0 17 Jan 2020
Sideways: Depth-Parallel Training of Video Models Mateusz Malinowski G. Swirszcz João Carreira Viorica Patraucean MDE 94 15 0 17 Jan 2020
Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent Giorgi Nadiradze Ilia Markov Bapi Chatterjee Vyacheslav Kungurtsev Dan Alistarh FedML 121 14 0 16 Jan 2020
Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation Madan Ravi Ganesh Jason J. Corso ODL 60 10 0 13 Jan 2020
Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey F. Sultana Abu Sufian P. Dutta SSeg 141 258 0 13 Jan 2020
Natural Image Matting via Guided Contextual Attention Yaoyi Li Hongtao Lu 82 169 0 13 Jan 2020
HyperSched: Dynamic Resource Reallocation for Model Development on a Deadline Richard Liaw Romil Bhardwaj Lisa Dunlap Yitian Zou Joseph E. Gonzalez Ion Stoica Alexey Tumanov 74 45 0 08 Jan 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 83 60 0 07 Jan 2020
Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis M. Chiu Xingqian Xu Yunchao Wei Zilong Huang Alex Schwing ... David Wilson Adrian Tudor N. Hovakimyan Thomas S. Huang Humphrey Shi 105 155 0 05 Jan 2020
EcoNAS: Finding Proxies for Economical Neural Architecture Search Dongzhan Zhou Xinchi Zhou Wenwei Zhang Chen Change Loy Shuai Yi Xuesen Zhang Wanli Ouyang 87 112 0 05 Jan 2020
Distributed Stochastic Algorithms for High-rate Streaming Principal Component Analysis Haroon Raja W. Bajwa 85 11 0 04 Jan 2020
Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks Zhaoxian Wu Qing Ling Tianyi Chen G. Giannakis FedML AAML 117 186 0 29 Dec 2019
Pipelined Training with Stale Weights of Deep Convolutional Neural Networks Lifu Zhang T. Abdelrahman 48 0 0 29 Dec 2019
Big Transfer (BiT): General Visual Representation Learning Alexander Kolesnikov Lucas Beyer Xiaohua Zhai J. Puigcerver Jessica Yung Sylvain Gelly N. Houlsby MQ 310 1,211 0 24 Dec 2019
Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild Xin Chen Lingxi Xie Jun Wu Qi Tian 137 96 0 23 Dec 2019
end-to-end training of a large vocabulary end-to-end speech recognition system Chanwoo Kim Sungsoo Kim Kwangyoun Kim Mehul Kumar Jiyeon Kim ... Eunhyang Kim Minkyoo Shin Shatrughan Singh Larry Heck Dhananjaya N. Gowda 61 27 0 22 Dec 2019
Towards Efficient Training for Neural Network Quantization Qing Jin Linjie Yang Zhenyu A. Liao MQ 110 42 0 21 Dec 2019
Learning Singing From Speech Liqiang Zhang Chengzhu Yu Heng Lu Chao Weng Yusong Wu Xiang Xie Zijin Li Dong Yu 53 8 0 20 Dec 2019
A Survey on Distributed Machine Learning Joost Verbraeken Matthijs Wolting Jonathan Katzy Jeroen Kloppenburg Tim Verbelen Jan S. Rellermeyer OOD 122 715 0 20 Dec 2019
AdaBits: Neural Network Quantization with Adaptive Bit-Widths Qing Jin Linjie Yang Zhenyu A. Liao MQ 93 124 0 20 Dec 2019
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 137 169 0 19 Dec 2019
MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Bo Li FedML 61 25 0 18 Dec 2019
PointRend: Image Segmentation as Rendering Alexander Kirillov Yuxin Wu Kaiming He Ross B. Girshick ISeg 185 904 0 17 Dec 2019
Direction Concentration Learning: Enhancing Congruency in Machine Learning Yan Luo Yongkang Wong Mohan Kankanhalli Qi Zhao 42 12 0 17 Dec 2019
UNAS: Differentiable Architecture Search Meets Reinforcement Learning Arash Vahdat Arun Mallya Ming-Yuan Liu Jan Kautz 85 34 0 16 Dec 2019
Dota 2 with Large Scale Deep Reinforcement Learning OpenAI OpenAI : Christopher Berner Greg Brockman Brooke Chan ... Szymon Sidor Ilya Sutskever Jie Tang Filip Wolski Susan Zhang GNN VLM CLL AI4CE LRM 181 1,840 0 13 Dec 2019
Local Context Normalization: Revisiting Local Normalization Anthony Ortiz Caleb Robinson Dan Morris O. Fuentes Christopher Kiekintveld Mahmudulla Hassan Nebojsa Jojic 44 26 0 12 Dec 2019