v1v2 (latest)

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

8 June 2017

Piotr Dollár

Papers citing "Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour"

50 / 2,054 papers shown

Title
Recent advances in deep learning theory Fengxiang He Dacheng Tao AI4CE 132 51 0 20 Dec 2020
Learning from History for Byzantine Robust Optimization Sai Praneeth Karimireddy Lie He Martin Jaggi FedML AAML 119 183 0 18 Dec 2020
Study on the Large Batch Size Training of Neural Networks Based on the Second Order Gradient Fengli Gao Huicai Zhong ODL 37 10 0 16 Dec 2020
Trex: Learning Execution Semantics from Micro-Traces for Binary Similarity Kexin Pei Zhou Xuan Junfeng Yang Suman Jana Baishakhi Ray 119 91 0 16 Dec 2020
GTA: Global Temporal Attention for Video Action Understanding Bo He Xitong Yang Zuxuan Wu Hao Chen Ser-Nam Lim Abhinav Shrivastava ViT 102 28 0 15 Dec 2020
$NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation framework$ NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation framework O. Hennigh S. Narasimhan M. A. Nabian Akshay Subramaniam Kaustubh Tangsali M. Rietmann J. Ferrandis Wonmin Byeon Z. Fang S. Choudhry PINN AI4CE 148 130 0 14 Dec 2020
Comparing the costs of abstraction for DL frameworks Maksim Levental Elena Orlova AI4CE 30 3 0 13 Dec 2020
Mask Guided Matting via Progressive Refinement Network Qihang Yu Jianming Zhang He Zhang Yilin Wang Zhe Lin N. Xu Yutong Bai Alan Yuille 87 115 0 12 Dec 2020
A Comprehensive Study of Deep Video Action Recognition Yi Zhu Xinyu Li Chunhui Liu Mohammadreza Zolfaghari Yuanjun Xiong Chongruo Wu Zhi-Li Zhang Joseph Tighe R. Manmatha Mu Li VLM AI4TS 129 188 0 11 Dec 2020
Cyclic orthogonal convolutions for long-range integration of features Federica Freddi Jezabel R. Garcia Michael Bromberg Sepehr Jalali Da-shan Shiu Alvin Chua A. Bernacchia 47 0 0 11 Dec 2020
Recent Theoretical Advances in Non-Convex Optimization Marina Danilova Pavel Dvurechensky Alexander Gasnikov Eduard A. Gorbunov Sergey Guminov Dmitry Kamzolov Innokentiy Shibaev 129 79 0 11 Dec 2020
Distributed Training of Graph Convolutional Networks using Subgraph Approximation Alexandra Angerd Keshav Balasubramanian M. Annavaram GNN 54 8 0 09 Dec 2020
Kernelized Classification in Deep Networks Sadeep Jayasumana Srikumar Ramalingam Sanjiv Kumar 43 4 0 08 Dec 2020
Parallel Training of Deep Networks with Local Updates Michael Laskin Luke Metz Seth Nabarrao Mark Saroufim Badreddine Noune Carlo Luschi Jascha Narain Sohl-Dickstein Pieter Abbeel FedML 122 27 0 07 Dec 2020
Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent Kangqiao Liu Liu Ziyin Masakuni Ueda MLT 152 40 0 07 Dec 2020
CARAFE++: Unified Content-Aware ReAssembly of FEatures Jiaqi Wang Kai-xiang Chen Rui Xu Ziwei Liu Chen Change Loy Dahua Lin 76 56 0 07 Dec 2020
When Do Curricula Work? Xiaoxia Wu Ethan Dyer Behnam Neyshabur 100 118 0 05 Dec 2020
Batch Group Normalization Xiao-Yun Zhou Jiacheng Sun Nanyang Ye Xu Lan Qijun Luo Bolin Lai P. Esperança Guang-Zhong Yang Zhenguo Li 158 17 0 04 Dec 2020
Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learning Haohang Xu Xiaopeng Zhang Hao Li Lingxi Xie H. Xiong Qi Tian SSL 60 12 0 04 Dec 2020
SAFCAR: Structured Attention Fusion for Compositional Action Recognition Tae Soo Kim Gregory Hager CoGe 67 10 0 03 Dec 2020
Accumulated Decoupled Learning: Mitigating Gradient Staleness in Inter-Layer Model Parallelization Huiping Zhuang Zhiping Lin Kar-Ann Toh 124 4 0 03 Dec 2020
Disentangling Label Distribution for Long-tailed Visual Recognition Youngkyu Hong Seungju Han Kwanghee Choi Seokjun Seo Beomsu Kim Buru Chang 89 238 0 01 Dec 2020
Diverse Temporal Aggregation and Depthwise Spatiotemporal Factorization for Efficient Video Classification Youngwan Lee Hyungil Kim Kimin Yun Jinyoung Moon 51 12 0 01 Dec 2020
Towards Better Accuracy-efficiency Trade-offs: Divide and Co-training Shuai Zhao Liguang Zhou Wenxiao Wang D. Cai Tin Lun Lam Yangsheng Xu 110 32 0 30 Nov 2020
Dynamic Curriculum Learning for Low-Resource Neural Machine Translation Chen Xu Bojie Hu Yufan Jiang Kai Feng Zeyang Wang Shen Huang Qi Ju Tong Xiao Jingbo Zhu 101 22 0 30 Nov 2020
Improving Layer-wise Adaptive Rate Methods using Trust Ratio Clipping Jeffrey Fong Siwei Chen Kaiqi Chen 33 2 0 27 Nov 2020
Grafit: Learning fine-grained image representations with coarse labels Hugo Touvron Alexandre Sablayrolles Matthijs Douze Matthieu Cord Hervé Jégou SSL 91 68 0 25 Nov 2020
No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems N. Sohoni Jared A. Dunnmon Geoffrey Angus Albert Gu Christopher Ré 90 252 0 25 Nov 2020
torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation Yoshitomo Matsubara 76 25 0 25 Nov 2020
Bringing AI To Edge: From Deep Learning's Perspective Di Liu Hao Kong Xiangzhong Luo Weichen Liu Ravi Subramaniam 116 125 0 25 Nov 2020
Multi-Domain Adversarial Feature Generalization for Person Re-Identification Shan Lin Chang-Tsun Li Alex C. Kot OOD 66 61 0 25 Nov 2020
A3D: Adaptive 3D Networks for Video Action Recognition Sijie Zhu Taojiannan Yang Matías Mendieta Chong Chen 3DH 70 13 0 24 Nov 2020
Adam $^+$ : A Stochastic Method with Adaptive Variance Reduction Mingrui Liu Wei Zhang Francesco Orabona Tianbao Yang 64 28 0 24 Nov 2020
Prior to Segment: Foreground Cues for Weakly Annotated Classes in Partially Supervised Instance Segmentation David Biertimpel Sindi Shkodrani A. S. Baslamisli N. Baka ISeg 70 1 0 23 Nov 2020
Scaling Wide Residual Networks for Panoptic Segmentation Liang-Chieh Chen Huiyu Wang Siyuan Qiao SSeg 134 49 0 23 Nov 2020
Distributed Deep Reinforcement Learning: An Overview Mohammad Reza Samsami Hossein Alimadad OffRL 43 27 0 22 Nov 2020
An Effective Anti-Aliasing Approach for Residual Networks C. N. Vasconcelos Hugo Larochelle Vincent Dumoulin Nicolas Le Roux Ross Goroshin SupR 70 32 0 20 Nov 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 397 4,087 0 20 Nov 2020
Contrastive Weight Regularization for Large Minibatch SGD Qiwei Yuan Weizhe Hua Yi Zhou Cunxi Yu OffRL 86 1 0 17 Nov 2020
EvoPose2D: Pushing the Boundaries of 2D Human Pose Estimation using Accelerated Neuroevolution with Weight Transfer William J. McNally Kanav Vats Alexander Wong J. McPhee 3DH 70 16 0 17 Nov 2020
Metastatic Cancer Image Classification Based On Deep Learning Method Guanwen Qiu Xiaobing Yu B. Sun Yunpeng Wang Lipei Zhang MedIm 18 1 0 13 Nov 2020
Distributed Sparse SGD with Majority Voting Kerem Ozfatura Emre Ozfatura Deniz Gunduz FedML 76 4 0 12 Nov 2020
An ensemble-based approach by fine-tuning the deep transfer learning models to classify pneumonia from chest X-ray images Sagar Kora Venu LM&MA 60 20 0 11 Nov 2020
Understanding Training Efficiency of Deep Learning Recommendation Models at Scale Bilge Acun Matthew Murphy Xiaodong Wang Jade Nie Carole-Jean Wu K. Hazelwood 95 113 0 11 Nov 2020
SALR: Sharpness-aware Learning Rate Scheduler for Improved Generalization Xubo Yue Maher Nouiehed Raed Al Kontar ODL 40 4 0 10 Nov 2020
Exploring the limits of Concurrency in ML Training on Google TPUs Sameer Kumar James Bradbury C. Young Yu Emma Wang Anselm Levskaya ... Tao Wang Tayo Oguntebi Yazhou Zu Yuanzhong Xu Andy Swing BDL AIMat MoE LRM 64 27 0 07 Nov 2020
Multi-task learning for electronic structure to predict and explore molecular potential energy surfaces Zhuoran Qiao Feizhi Ding Matthew Welborn P. J. Bygrave Daniel G. A. Smith Anima Anandkumar F. Manby Thomas F. Miller 62 7 0 05 Nov 2020
Direction Matters: On the Implicit Bias of Stochastic Gradient Descent with Moderate Learning Rate Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu 102 18 0 04 Nov 2020
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding Mike Roberts Jason Ramapuram Anurag Ranjan Atulit Kumar Miguel Angel Bautista Nathan Paczan Russ Webb Joshua M. Susskind 197 393 0 04 Nov 2020
Reverse engineering learned optimizers reveals known and novel mechanisms Niru Maheswaranathan David Sussillo Luke Metz Ruoxi Sun Jascha Narain Sohl-Dickstein 101 22 0 04 Nov 2020