Large Batch Training of Convolutional Networks

13 August 2017

Yang You

Igor Gitman

Boris Ginsburg

ODL

ArXiv PDF HTML

Papers citing "Large Batch Training of Convolutional Networks"

50 / 545 papers shown

Title
Scaling the Wild: Decentralizing Hogwild!-style Shared-memory SGD Bapi Chatterjee Vyacheslav Kungurtsev Dan Alistarh FedML 27 2 0 13 Mar 2022
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer Greg Yang J. E. Hu Igor Babuschkin Szymon Sidor Xiaodong Liu David Farhi Nick Ryder J. Pachocki Weizhu Chen Jianfeng Gao 31 149 0 07 Mar 2022
Measuring Self-Supervised Representation Quality for Downstream Classification using Discriminative Features Neha Kalibhat Kanika Narang Hamed Firooz Maziar Sanjabi S. Feizi SSL 40 7 0 03 Mar 2022
Audio Self-supervised Learning: A Survey Shuo Liu Adria Mallol-Ragolta Emilia Parada-Cabeleiro Kun Qian Xingshuo Jing Alexander Kathan Bin Hu Bjoern W. Schuller SSL 45 106 0 02 Mar 2022
FastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours Shenggan Cheng Xuanlei Zhao Guangyang Lu Bin-Rui Li Zhongming Yu Tian Zheng R. Wu Xiwen Zhang Jian Peng Yang You AI4CE 27 30 0 02 Mar 2022
SWIS: Self-Supervised Representation Learning For Writer Independent Offline Signature Verification Siladittya Manna Soumitri Chattopadhyay Saumik Bhattacharya Umapada Pal 19 12 0 26 Feb 2022
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance Zhuoning Yuan Yuexin Wu Zi-qi Qiu Xianzhi Du Lijun Zhang Denny Zhou Tianbao Yang 36 26 0 24 Feb 2022
Survey on Large Scale Neural Network Training Julia Gusak Daria Cherniuk Alena Shilova A. Katrutsa Daniel Bershatsky ... Lionel Eyraud-Dubois Oleg Shlyazhko Denis Dimitrov Ivan Oseledets Olivier Beaumont 41 10 0 21 Feb 2022
A Self-Supervised Descriptor for Image Copy Detection Ed Pizzi Sreya . Dutta Roy Sugosh Nagavara Ravindra Priya Goyal Matthijs Douze SSL 34 117 0 21 Feb 2022
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision Priya Goyal Quentin Duval Isaac Seessel Mathilde Caron Ishan Misra Levent Sagun Armand Joulin Piotr Bojanowski VLM SSL 28 110 0 16 Feb 2022
Orthogonalising gradients to speed up neural network optimisation Mark Tuddenham Adam Prugel-Bennett Jonathan Hare ODL 15 2 0 14 Feb 2022
Conditional Contrastive Learning with Kernel Yao-Hung Hubert Tsai Tianqi Li Martin Q. Ma Han Zhao Kun Zhang Louis-Philippe Morency Ruslan Salakhutdinov 27 25 0 11 Feb 2022
Context Autoencoder for Self-Supervised Representation Learning Xiaokang Chen Mingyu Ding Xiaodi Wang Ying Xin Shentong Mo Yunhao Wang Shumin Han Ping Luo Gang Zeng Jingdong Wang SSL 47 386 0 07 Feb 2022
Harmony: Overcoming the Hurdles of GPU Memory Capacity to Train Massive DNN Models on Commodity Servers Youjie Li Amar Phanishayee D. Murray Jakub Tarnawski Nam Sung Kim 19 19 0 02 Feb 2022
Distributed SLIDE: Enabling Training Large Neural Networks on Low Bandwidth and Simple CPU-Clusters via Model Parallelism and Sparsity Minghao Yan Nicholas Meisburger Tharun Medini Anshumali Shrivastava 19 6 0 29 Jan 2022
You Only Cut Once: Boosting Data Augmentation with a Single Cut Junlin Han Pengfei Fang Weihong Li Jie Hong M. Armin Ian Reid L. Petersson Hongdong Li 42 27 0 28 Jan 2022
Existence and Estimation of Critical Batch Size for Training Generative Adversarial Networks with Two Time-Scale Update Rule Naoki Sato Hideaki Iiduka EGVM 20 4 0 28 Jan 2022
Revisiting RCAN: Improved Training for Image Super-Resolution Zudi Lin Prateek Garg Atmadeep Banerjee Salma Abdel Magid Deqing Sun Yulun Zhang Luc Van Gool D. Wei Hanspeter Pfister SupR 53 57 0 27 Jan 2022
Neural Manifold Clustering and Embedding Zengyi Li Yubei Chen Yann LeCun Friedrich T. Sommer DRL 22 42 0 24 Jan 2022
Boundary-aware Self-supervised Learning for Video Scene Segmentation Jonghwan Mun Minchul Shin Gunsoo Han Sangho Lee S. Ha Joonseok Lee Eun-Sol Kim SSL 59 20 0 14 Jan 2022
Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet? Nenad Tomašev Ioana Bica Brian McWilliams Lars Buesing Razvan Pascanu Charles Blundell Jovana Mitrović SSL 92 81 0 13 Jan 2022
Representation Learning via Consistent Assignment of Views to Clusters T. Silva Adín Ramirez Rivera SSL 19 8 0 31 Dec 2021
Adversarial Learning for Incentive Optimization in Mobile Payment Marketing Xuanying Chen Zhining Liu Li Yu Sen Li Lihong Gu Xiaodong Zeng Yize Tan Jinjie Gu OffRL 14 5 0 28 Dec 2021
Max-Margin Contrastive Learning Anshul B. Shah S. Sra Ramalingam Chellappa A. Cherian SSL 37 44 0 21 Dec 2021
Efficient Strong Scaling Through Burst Parallel Training S. Park Joshua Fried Sunghyun Kim Mohammad Alizadeh Adam Belay GNN LRM 32 10 0 19 Dec 2021
Contrastive Vision-Language Pre-training with Limited Resources Quan Cui Boyan Zhou Yu Guo Weidong Yin Hao Wu Osamu Yoshie Yubo Chen VLM CLIP 19 33 0 17 Dec 2021
Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer Yanpeng Zhao Jack Hessel Youngjae Yu Ximing Lu Rowan Zellers Yejin Choi 25 27 0 16 Dec 2021
Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration Yu Wang Jingyang Lin Jingjing Zou Yingwei Pan Ting Yao Tao Mei OODD 34 12 0 15 Dec 2021
Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications Karin Stacke Jonas Unger Claes Lundström Gabriel Eilertsen OOD SSL 26 25 0 10 Dec 2021
Self-Supervised Models are Continual Learners Enrico Fini Victor G. Turrisi da Costa Xavier Alameda-Pineda Elisa Ricci Alahari Karteek Julien Mairal BDL CLL SSL 55 161 0 08 Dec 2021
Unsupervised Representation Learning via Neural Activation Coding Yookoon Park Sangho Lee Gunhee Kim David M. Blei SSL 26 8 0 07 Dec 2021
A Fast Knowledge Distillation Framework for Visual Recognition Zhiqiang Shen Eric P. Xing VLM 14 46 0 02 Dec 2021
Similarity Contrastive Estimation for Self-Supervised Soft Contrastive Learning J. Denize Jaonary Rabarisoa Astrid Orcesi Romain Hérault S. Canu SSL 21 30 0 29 Nov 2021
ViCE: Improving Dense Representation Learning by Superpixelization and Contrasting Cluster Assignment Robin Karlsson Tomoki Hayashi Keisuke Fujii Alexander Carballo Kento Ohtani K. Takeda SSL 42 4 0 24 Nov 2021
RegionCL: Can Simple Region Swapping Contribute to Contrastive Learning? Yufei Xu Qiming Zhang Jing Zhang Dacheng Tao SSL 21 18 0 24 Nov 2021
MIO : Mutual Information Optimization using Self-Supervised Binary Contrastive Learning Siladittya Manna Umapada Pal Saumik Bhattacharya SSL 35 1 0 24 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 322 7,503 0 11 Nov 2021
Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicity Pritam Sarkar Ali Etemad SSL 36 11 0 09 Nov 2021
MixSiam: A Mixture-based Approach to Self-supervised Representation Learning Xiaoyang Guo Tianhao Zhao Yutian Lin Bo Du SSL 30 6 0 04 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 32 14 0 01 Nov 2021
Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance Nathanaël Fijalkow DoangJoo Synn Jooyoung Park Jong-Kook Kim 30 5 0 24 Oct 2021
MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems S. Farrell M. Emani J. Balma L. Drescher Aleksandr Drozd ... Akihiro Tabuchi V. Vishwanath Mohamed Wahib Masafumi Yamazaki Junqi Yin VLM 37 35 0 21 Oct 2021
A Unified and Refined Convergence Analysis for Non-Convex Decentralized Learning Sulaiman A. Alghunaim Kun Yuan 42 57 0 19 Oct 2021
DECAR: Deep Clustering for learning general-purpose Audio Representations Sreyan Ghosh Sandesh V Katta Ashish Seth S. Umesh SSL 36 12 0 17 Oct 2021
Unsupervised Representation Learning for Binary Networks by Joint Classifier Learning Dahyun Kim Jonghyun Choi SSL MQ 25 5 0 17 Oct 2021
Trade-offs of Local SGD at Scale: An Empirical Study Jose Javier Gonzalez Ortiz Jonathan Frankle Michael G. Rabbat Ari S. Morcos Nicolas Ballas FedML 43 19 0 15 Oct 2021
Self-Supervised Learning by Estimating Twin Class Distributions Feng Wang Tao Kong Rufeng Zhang Huaping Liu Hang Li SSL 60 18 0 14 Oct 2021
Decoupled Contrastive Learning Chun-Hsiao Yeh Cheng-Yao Hong Yen-Chi Hsu Tyng-Luh Liu Yubei Chen Yann LeCun 183 183 0 13 Oct 2021
Towards Demystifying Representation Learning with Non-contrastive Self-supervision Xiang Wang Xinlei Chen S. Du Yuandong Tian SSL 21 26 0 11 Oct 2021
Weakly Supervised Contrastive Learning Mingkai Zheng Fei Wang Shan You Chao Qian Changshui Zhang Xiaogang Wang Chang Xu SSL 45 121 0 10 Oct 2021