Large Batch Training of Convolutional Networks

13 August 2017

Yang You

Igor Gitman

Boris Ginsburg

ODL

ArXiv PDF HTML

Papers citing "Large Batch Training of Convolutional Networks"

50 / 545 papers shown

Title
Temperature as Uncertainty in Contrastive Learning Oliver Zhang Mike Wu Jasmine Bayrooti Noah D. Goodman UQCV 22 30 0 08 Oct 2021
Unsupervised Selective Labeling for More Effective Semi-Supervised Learning Xudong Wang Long Lian Stella X. Yu 194 33 0 06 Oct 2021
On the Surrogate Gap between Contrastive and Supervised Losses Han Bao Yoshihiro Nagano Kento Nozawa SSL UQCV 46 19 0 06 Oct 2021
DualNet: Continual Learning, Fast and Slow Quang Pham Chenghao Liu Guosheng Lin CLL 77 44 0 01 Oct 2021
UFO-ViT: High Performance Linear Vision Transformer without Softmax Jeonggeun Song ViT 114 20 0 29 Sep 2021
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 89 72 0 29 Sep 2021
Compressive Visual Representations Kuang-Huei Lee Anurag Arnab S. Guadarrama John F. Canny Ian S. Fischer SSL 65 48 0 27 Sep 2021
MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving Kai Chen Lanqing Hong Hang Xu Zhenguo Li Dit-Yan Yeung SSL 13 55 0 27 Aug 2021
Learning From Long-Tailed Data With Noisy Labels Shyamgopal Karthik Jérôme Revaud Boris Chidlovskii SSL NoLa 22 27 0 25 Aug 2021
ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning Zhiwu Qing Ziyuan Huang Shiwei Zhang Mingqian Tang Changxin Gao M. Ang Ronglei Ji Nong Sang 45 3 0 24 Aug 2021
Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations Josh Beal Hao Wu Dong Huk Park Andrew Zhai Dmitry Kislyuk ViT 23 29 0 12 Aug 2021
Logit Attenuating Weight Normalization Aman Gupta R. Ramanath Jun Shi Anika Ramachandran Sirou Zhou Mingzhou Zhou S. Keerthi 45 1 0 12 Aug 2021
Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusters Chen Sun Shenggui Li Jinyue Wang Jun Yu 54 47 0 08 Aug 2021
A Low Rank Promoting Prior for Unsupervised Contrastive Learning Yu Wang Jingyang Lin Qi Cai Yingwei Pan Ting Yao Hongyang Chao Tao Mei SSL 38 16 0 05 Aug 2021
Self-Supervised Feature Learning of 1D Convolutional Neural Networks with Contrastive Loss for Eating Detection Using an In-Ear Microphone Vasileios Papapanagiotou Christos Diou A. Delopoulos SSL 29 6 0 02 Aug 2021
Automated Learning Rate Scheduler for Large-batch Training Chiheon Kim Saehoon Kim Jongmin Kim Donghoon Lee Sungwoong Kim 14 18 0 13 Jul 2021
REX: Revisiting Budgeted Training with an Improved Schedule John Chen Cameron R. Wolfe Anastasios Kyrillidis 15 8 0 09 Jul 2021
Distance-based Hyperspherical Classification for Multi-source Open-Set Domain Adaptation S. Bucci Francesco Cappio Borlino Barbara Caputo Tatiana Tommasi VLM 35 14 0 05 Jul 2021
Unsupervised Model Drift Estimation with Batch Normalization Statistics for Dataset Shift Detection and Model Selection Won-Jo Lee Seokhyun Byun Jooeun Kim Minje Park Kirill Chechil AI4TS 21 2 0 01 Jul 2021
What can linear interpolation of neural network loss landscapes tell us? Tiffany J. Vlaar Jonathan Frankle MoMe 30 27 0 30 Jun 2021
Self-Contrastive Learning: Single-viewed Supervised Contrastive Framework using Sub-network Sangmin Bae Sungnyun Kim Jongwoo Ko Gihun Lee SeungJong Noh Se-Young Yun SSL 28 6 0 29 Jun 2021
Robust Distributed Optimization With Randomly Corrupted Gradients Berkay Turan César A. Uribe Hoi-To Wai M. Alizadeh 23 16 0 28 Jun 2021
High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails Ashok Cutkosky Harsh Mehta 30 58 0 28 Jun 2021
Feature Alignment as a Generative Process T. S. Farias Jonas Maziero DiffM BDL 28 1 0 23 Jun 2021
SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Driving Jianhua Han Xiwen Liang Hang Xu Kai Chen Lanqing Hong ... Chao Ye Wei Zhang Zhenguo Li Xi Liang Chunjing Xu 16 77 0 21 Jun 2021
CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation Enda Yu Dezun Dong Yemao Xu Shuo Ouyang Xiangke Liao 16 5 0 21 Jun 2021
Multirate Training of Neural Networks Tiffany J. Vlaar B. Leimkuhler 27 4 0 20 Jun 2021
Distributed Deep Learning in Open Collaborations Michael Diskin Alexey Bukhtiyarov Max Ryabinin Lucile Saulnier Quentin Lhoest ... Denis Mazur Ilia Kobelev Yacine Jernite Thomas Wolf Gennady Pekhimenko FedML 41 54 0 18 Jun 2021
Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrasting Martine Toering Ioannis Gatopoulos M. Stol Vincent Tao Hu SSL 40 11 0 18 Jun 2021
On Large-Cohort Training for Federated Learning Zachary B. Charles Zachary Garrett Zhouyuan Huo Sergei Shmulyian Virginia Smith FedML 21 113 0 15 Jun 2021
NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning Minghan Yang Dong Xu Qiwen Cui Zaiwen Wen Pengxiang Xu 26 4 0 14 Jun 2021
On the Convergence of Differentially Private Federated Learning on Non-Lipschitz Objectives, and with Normalized Client Updates Rudrajit Das Abolfazl Hashemi Sujay Sanghavi Inderjit S. Dhillon FedML 48 4 0 13 Jun 2021
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 38 290 0 11 Jun 2021
PeCLR: Self-Supervised 3D Hand Pose Estimation from monocular RGB via Equivariant Contrastive Learning Adrian Spurr Aneesh Dahiya Xi Wang Xucong Zhang Otmar Hilliges SSL 3DH 36 2 0 10 Jun 2021
A Generalizable Approach to Learning Optimizers Diogo Almeida Clemens Winter Jie Tang Wojciech Zaremba AI4CE 19 29 0 02 Jun 2021
Tesseract: Parallelize the Tensor Parallelism Efficiently Boxiang Wang Qifan Xu Zhengda Bian Yang You VLM GNN 9 30 0 30 May 2021
Maximizing Parallelism in Distributed Training for Huge Neural Networks Zhengda Bian Qifan Xu Boxiang Wang Yang You MoE 14 46 0 30 May 2021
Deep Learning for EEG Seizure Detection in Preterm Infants Alison O'Shea Rehan Ahmed G. Lightbody S. Mathieson Elena Pavlidis Rhodri O Lloyd F. Pisani W. Marnane Geraldine Boylan A. Temko 18 30 0 28 May 2021
Heterogeneous Contrastive Learning Lecheng Zheng Jinjun Xiong Yada Zhu Jingrui He 40 21 0 19 May 2021
Layerwise Optimization by Gradient Decomposition for Continual Learning Shixiang Tang Dapeng Chen Jinguo Zhu Shijie Yu Wanli Ouyang CLL 24 63 0 17 May 2021
When Does Contrastive Visual Representation Learning Work? Elijah Cole Xuan S. Yang Kimberly Wilber Oisin Mac Aodha Serge Belongie SSL 26 123 0 12 May 2021
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning Adrien Bardes Jean Ponce Yann LeCun SSL DML 86 906 0 11 May 2021
On Feature Decorrelation in Self-Supervised Learning Tianyu Hua Wenxiao Wang Zihui Xue Sucheng Ren Yue Wang Hang Zhao SSL OOD 140 188 0 02 May 2021
A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning Christoph Feichtenhofer Haoqi Fan Bo Xiong Ross B. Girshick Kaiming He SSL AI4TS 39 257 0 29 Apr 2021
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations Debidatta Dwibedi Y. Aytar Jonathan Tompson P. Sermanet Andrew Zisserman SSL 191 457 0 29 Apr 2021
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples Mahmoud Assran Mathilde Caron Ishan Misra Piotr Bojanowski Armand Joulin Nicolas Ballas Michael G. Rabbat SSL 29 147 0 28 Apr 2021
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training Kun Yuan Yiming Chen Xinmeng Huang Yingya Zhang Pan Pan Yinghui Xu W. Yin MoE 60 61 0 24 Apr 2021
Contrastive Learning Improves Model Robustness Under Label Noise Aritra Ghosh Andrew Lan NoLa 21 58 0 19 Apr 2021
Learning Metrics from Mean Teacher: A Supervised Learning Method for Improving the Generalization of Speaker Verification System Ju-ho Kim Hye-jin Shim Jee-weon Jung Ha-Jin Yu 22 1 0 14 Apr 2021
Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation Kyeongpil Kang Kyohoon Jin Soyoung Yang Show-Ling Jang Jaegul Choo Yougbin Kim MU 11 16 0 13 Apr 2021