Large Batch Training of Convolutional Networks

13 August 2017

Yang You

Igor Gitman

Boris Ginsburg

ODL

ArXiv PDF HTML

Papers citing "Large Batch Training of Convolutional Networks"

50 / 545 papers shown

Title
An Empirical Study of Training Self-Supervised Vision Transformers Xinlei Chen Saining Xie Kaiming He ViT 75 1,811 0 05 Apr 2021
Pre-training strategies and datasets for facial representation learning Adrian Bulat Shiyang Cheng Jing Yang A. Garbett Enrique Sanchez Georgios Tzimiropoulos CVBM SSL 29 36 0 30 Mar 2021
Embedding Transfer with Label Relaxation for Improved Metric Learning Sungyeon Kim Dongwon Kim Minsu Cho Suha Kwak 32 32 0 27 Mar 2021
Contrastive Domain Adaptation Mamatha Thota Georgios Leontidis SSL 32 59 0 26 Mar 2021
Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations Chaitanya K. Ryali D. Schwab Ari S. Morcos SSL 37 9 0 23 Mar 2021
Self-supervised Representation Learning with Relative Predictive Coding Yao-Hung Hubert Tsai Martin Q. Ma Muqiao Yang Han Zhao Louis-Philippe Morency Ruslan Salakhutdinov SSL AI4TS 38 36 0 21 Mar 2021
Self-Supervised Classification Network Elad Amrani Leonid Karlinsky A. Bronstein SSL 27 28 0 19 Mar 2021
Efficient Visual Pretraining with Contrastive Detection Olivier J. Hénaff Skanda Koppula Jean-Baptiste Alayrac Aaron van den Oord Oriol Vinyals João Carreira VLM SSL 38 162 0 19 Mar 2021
UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning Zhigang Dai Bolun Cai Yugeng Lin Junying Chen SSL 33 6 0 19 Mar 2021
2D histology meets 3D topology: Cytoarchitectonic brain mapping with Graph Neural Networks Christian Schiffer Stefan Harmeling Katrin Amunts Timo Dickscheid 3DH 3DV 21 2 0 09 Mar 2021
Extending Contrastive Learning to Unsupervised Coreset Selection Jeongwoo Ju Heechul Jung Yoonju Oh Junmo Kim SSL 19 12 0 05 Mar 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 81 2,299 0 04 Mar 2021
Self-supervised Pretraining of Visual Features in the Wild Priya Goyal Mathilde Caron Benjamin Lefaudeux Min Xu Pengchao Wang ... Mannat Singh Vitaliy Liptchinsky Ishan Misra Armand Joulin Piotr Bojanowski VLM SSL 27 271 0 02 Mar 2021
Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning A. Jaus Kailun Yang Rainer Stiefelhagen 42 36 0 01 Mar 2021
Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory Jason Ramapuram Yan Wu Alexandros Kalousis 12 4 0 20 Feb 2021
Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learning Kento Nozawa Issei Sato SSL 22 43 0 13 Feb 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 138 281 0 12 Feb 2021
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes Zachary Nado Justin M. Gilmer Christopher J. Shallue Rohan Anil George E. Dahl ODL 22 27 0 12 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 223 512 0 11 Feb 2021
Supervised Tree-Wasserstein Distance Yuki Takezawa Ryoma Sato M. Yamada 24 12 0 27 Jan 2021
Supervised Momentum Contrastive Learning for Few-Shot Classification Orchid Majumder Avinash Ravichandran Subhransu Maji Alessandro Achille M. Polito Stefano Soatto SSL 21 12 0 26 Jan 2021
Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning Lang Huang Chaoning Zhang Hongyang R. Zhang SSL 33 24 0 21 Jan 2021
Momentum^2 Teacher: Momentum Teacher with Momentum Statistics for Self-Supervised Learning Zeming Li Songtao Liu Jian Sun 51 16 0 19 Jan 2021
Self-Supervised Representation Learning from Flow Equivariance Yuwen Xiong Mengye Ren Wenyuan Zeng R. Urtasun SSL 29 28 0 16 Jan 2021
Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration Congliang Chen Li Shen Fangyu Zou Wei Liu 46 29 0 14 Jan 2021
Big Self-Supervised Models Advance Medical Image Classification Shekoofeh Azizi Basil Mustafa Fiona Ryan Zach Beaver Jan Freyberg ... Alan Karthikesalingam Simon Kornblith Ting-Li Chen Vivek Natarajan Mohammad Norouzi SSL 48 506 0 13 Jan 2021
Take More Positives: An Empirical Study of Contrastive Learing in Unsupervised Person Re-Identification Xuanyu He Wei Emma Zhang Ran Song Qian Zhang X. Lan Lin Ma 41 1 0 12 Jan 2021
Crossover-SGD: A gossip-based communication in distributed deep learning for alleviating large mini-batch problem and enhancing scalability Sangho Yeo Minho Bae Minjoong Jeong Oh-Kyoung Kwon Sangyoon Oh 13 3 0 30 Dec 2020
Data optimization for large batch distributed training of deep neural networks Shubhankar Gahlot Junqi Yin Mallikarjun Shankar 21 1 0 16 Dec 2020
Study on the Large Batch Size Training of Neural Networks Based on the Second Order Gradient Fengli Gao Huicai Zhong ODL 20 9 0 16 Dec 2020
Self supervised contrastive learning for digital histopathology Ozan Ciga Tony Xu Anne L. Martel SSL 114 307 0 27 Nov 2020
Improving Layer-wise Adaptive Rate Methods using Trust Ratio Clipping Jeffrey Fong Siwei Chen Kaiqi Chen 8 2 0 27 Nov 2020
Contrastive Representation Learning for Whole Brain Cytoarchitectonic Mapping in Histological Human Brain Sections Christian Schiffer Katrin Amunts Stefan Harmeling Timo Dickscheid 22 11 0 25 Nov 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 52 3,967 0 20 Nov 2020
Data-efficient Alignment of Multimodal Sequences by Aligning Gradient Updates and Internal Feature Distributions Jianan Wang Boyang Albert Li Xiangyu Fan Jing-Hua Lin Yanwei Fu 25 2 0 15 Nov 2020
Towards Domain-Agnostic Contrastive Learning Vikas Verma Minh-Thang Luong Kenji Kawaguchi Hieu H. Pham Quoc V. Le SSL 15 116 0 09 Nov 2020
Exploring the limits of Concurrency in ML Training on Google TPUs Sameer Kumar James Bradbury C. Young Yu Emma Wang Anselm Levskaya ... Tao Wang Tayo Oguntebi Yazhou Zu Yuanzhong Xu Andy Swing BDL AIMat MoE LRM 25 27 0 07 Nov 2020
Intriguing Properties of Contrastive Losses Ting Chen Calvin Luo Lala Li 36 174 0 05 Nov 2020
A Survey on Contrastive Self-supervised Learning Ashish Jaiswal Ashwin Ramesh Babu Mohammad Zaki Zadeh Debapriya Banerjee F. Makedon SSL 51 1,360 0 31 Oct 2020
Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour Arissa Wongpanich Hieu H. Pham J. Demmel Mingxing Tan Quoc V. Le Yang You Sameer Kumar 11 8 0 30 Oct 2020
Stochastic Optimization with Laggard Data Pipelines Naman Agarwal Rohan Anil Tomer Koren Kunal Talwar Cyril Zhang 13 12 0 26 Oct 2020
Identifying Learning Rules From Neural Network Observables Aran Nayebi S. Srivastava Surya Ganguli Daniel L. K. Yamins 16 21 0 22 Oct 2020
How much progress have we made in neural network training? A New Evaluation Protocol for Benchmarking Optimizers Yuanhao Xiong Xuanqing Liu Li-Cheng Lan Yang You Si Si Cho-Jui Hsieh OOD 17 1 0 19 Oct 2020
CLAR: Contrastive Learning of Auditory Representations Haider Al-Tahan Y. Mohsenzadeh SSL 118 56 0 19 Oct 2020
Improving Transformation Invariance in Contrastive Representation Learning Adam Foster Rattana Pukdee Tom Rainforth 53 23 0 19 Oct 2020
Representation Learning via Invariant Causal Mechanisms Jovana Mitrović Brian McWilliams Jacob Walker Lars Buesing Charles Blundell CML OOD SSL 31 247 0 15 Oct 2020
EqCo: Equivalent Rules for Self-supervised Contrastive Learning Benjin Zhu Junqiang Huang Zeming Li Xiangyu Zhang Jian Sun SSL 22 13 0 05 Oct 2020
Normalization Techniques in Training DNNs: Methodology, Analysis and Application Lei Huang Jie Qin Yi Zhou Fan Zhu Li Liu Ling Shao AI4CE 12 255 0 27 Sep 2020
G-SimCLR : Self-Supervised Contrastive Learning with Guided Projection via Pseudo Labelling Souradip Chakraborty A. R. Gosthipaty Sayak Paul SSL 13 16 0 25 Sep 2020
A Framework For Contrastive Self-Supervised Learning And Designing A New Approach William Falcon Kyunghyun Cho SSL 21 103 0 31 Aug 2020