Large Batch Training of Convolutional Networks

13 August 2017

Yang You

Igor Gitman

Boris Ginsburg

ODL

ArXiv PDF HTML

Papers citing "Large Batch Training of Convolutional Networks"

50 / 545 papers shown

Title
Personalized Food Image Classification: Benchmark Datasets and New Baseline Xinyue Pan Jiangpeng He Fengqing M Zhu 65 5 0 15 Sep 2023
Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit? Bill Psomas Ioannis Kakogeorgiou Konstantinos Karantzalos Yannis Avrithis ViT 38 8 0 13 Sep 2023
Decoupling Common and Unique Representations for Multimodal Self-supervised Learning Yi Wang C. Albrecht Nassim Ait Ali Braham Chenying Liu Zhitong Xiong Xiaoxiang Zhu SSL 25 16 0 11 Sep 2023
Representation Synthesis by Probabilistic Many-Valued Logic Operation in Self-Supervised Learning Hiroki Nakamura Masashi Okada T. Taniguchi SSL NAI 31 0 0 08 Sep 2023
A Survey on Self-Supervised Representation Learning Tobias Uelwer Jan Robine Stefan Sylvius Wagner Marc Höftmann Eric Upschulte S. Konietzny Maike Behrendt Stefan Harmeling SSL AI4TS OOD 29 12 0 22 Aug 2023
CoNe: Contrast Your Neighbours for Supervised Image Classification Mingkai Zheng Shan You Lang Huang Xiu Su Fei Wang Chao Qian Xiaogang Wang Chang Xu VLM 26 0 0 21 Aug 2023
Improving Pixel-based MIM by Reducing Wasted Modeling Capability Yuan Liu Songyang Zhang Jiacheng Chen Zhaohui Yu Kai-xiang Chen Dahua Lin 29 29 0 01 Aug 2023
Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum Amnon Geifman Daniel Barzilai Ronen Basri Meirav Galun 29 5 0 26 Jul 2023
The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learning Borja Rodríguez Gálvez Arno Blaas P. Rodríguez Adam Goliñski Xavier Suau Jason Ramapuram Dan Busbridge Luca Zappella 50 6 0 20 Jul 2023
DreamTeacher: Pretraining Image Backbones with Deep Generative Models Daiqing Li Huan Ling Amlan Kar David Acuna Seung Wook Kim Karsten Kreis Antonio Torralba Sanja Fidler VLM DiffM 22 27 0 14 Jul 2023
Multiplicative update rules for accelerating deep learning training and increasing robustness Manos Kirtas Nikolaos Passalis Anastasios Tefas AAML OOD 36 2 0 14 Jul 2023
Mini-Batch Optimization of Contrastive Loss Jaewoong Cho Kartik K. Sreenivasan Keon Lee Kyunghoo Mun Soheun Yi Jeong-Gwan Lee Anna Lee Jy-yong Sohn Dimitris Papailiopoulos Kangwook Lee SSL 43 7 0 12 Jul 2023
Self-Supervised Learning with Lie Symmetries for Partial Differential Equations Grégoire Mialon Q. Garrido Hannah Lawrence Danyal Rehman Yann LeCun B. Kiani SSL 35 26 0 11 Jul 2023
CAME: Confidence-guided Adaptive Memory Efficient Optimization Yang Luo Xiaozhe Ren Zangwei Zheng Zhuo Jiang Xin Jiang Yang You ODL 23 17 0 05 Jul 2023
Multi-network Contrastive Learning Based on Global and Local Representations Weiquan Li Xianzhong Long Yun Li SSL 28 0 0 28 Jun 2023
Scaling MLPs: A Tale of Inductive Bias Gregor Bachmann Sotiris Anagnostidis Thomas Hofmann 40 38 0 23 Jun 2023
Deep Fusion: Efficient Network Training via Pre-trained Initializations Hanna Mazzawi X. Gonzalvo Michael Wunder Sammy Jerome Benoit Dherin AI4CE 39 3 0 20 Jun 2023
DropCompute: simple and more robust distributed synchronous training via compute variance reduction Niv Giladi Shahar Gottlieb Moran Shkolnik A. Karnieli Ron Banner Elad Hoffer Kfir Y. Levy Daniel Soudry 38 2 0 18 Jun 2023
ViP: A Differentially Private Foundation Model for Computer Vision Yaodong Yu Maziar Sanjabi Yi Ma Kamalika Chaudhuri Chuan Guo 24 12 0 15 Jun 2023
$$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning$ $\textbf{A}^2\textbf{CiD}^2$ : Accelerating Asynchronous Communication in Decentralized Deep Learning Adel Nabli Eugene Belilovsky Edouard Oyallon 24 6 0 14 Jun 2023
Semi-supervised learning made simple with self-supervised clustering Enrico Fini Pietro Astolfi Alahari Karteek Xavier Alameda-Pineda Julien Mairal Moin Nabi Elisa Ricci SSL 39 26 0 13 Jun 2023
Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning Sungmin Cha Kyunghyun Cho Taesup Moon BDL CLL 38 2 0 08 Jun 2023
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent Tongtian Zhu Fengxiang He Kaixuan Chen Mingli Song Dacheng Tao 34 15 0 05 Jun 2023
DeepVAT: A Self-Supervised Technique for Cluster Assessment in Image Datasets Alokendu Mazumder Tirthajit Baruah A. Singh Pagadala Krishna Murthy Vishwajeet Pattanaik Punit Rathore 13 0 0 29 May 2023
Intelligent gradient amplification for deep neural networks S. Basodi K. Pusuluri Xueli Xiao Yi Pan ODL 21 1 0 29 May 2023
Fine-Tuning Language Models with Just Forward Passes Sadhika Malladi Tianyu Gao Eshaan Nichani Alexandru Damian Jason D. Lee Danqi Chen Sanjeev Arora 43 180 0 27 May 2023
Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks Atli Kosson Bettina Messmer Martin Jaggi 35 12 0 26 May 2023
SING: A Plug-and-Play DNN Learning Technique Adrien Courtois Damien Scieur Jean-Michel Morel Pablo Arias Thomas Eboli 36 0 0 25 May 2023
READ: Recurrent Adaptation of Large Transformers Sida I. Wang John Nguyen Ke Li Carole-Jean Wu 28 11 0 24 May 2023
Delving Deeper into Data Scaling in Masked Image Modeling Cheng Lu Xiaojie Jin Qibin Hou Jun Hao Liew Mingg-Ming Cheng Jiashi Feng 38 4 0 24 May 2023
Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization Zimeng Qiu Quanqi Hu Zhuoning Yuan Denny Zhou Lijun Zhang Tianbao Yang 34 17 0 19 May 2023
Tuned Contrastive Learning Chaitanya Animesh Manmohan Chandraker SSL 19 0 0 18 May 2023
Revisiting the Minimalist Approach to Offline Reinforcement Learning Denis Tarasov Vladislav Kurenkov Alexander Nikulin Sergey Kolesnikov OffRL 33 37 0 16 May 2023
Medical supervised masked autoencoders: Crafting a better masking strategy and efficient fine-tuning schedule for medical image classification Jia-ju Mao Shu-Hua Guo Yuan Chang Xuesong Yin Binling Nie 28 2 0 10 May 2023
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch Kazuki Osawa Satoki Ishikawa Rio Yokota Shigang Li Torsten Hoefler ODL 46 14 0 08 May 2023
PGrad: Learning Principal Gradients For Domain Generalization Zhe Wang J. E. Grigsby Yanjun Qi OOD 29 10 0 02 May 2023
SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentation Subhajit Maity Sanket Biswas Siladittya Manna Ayan Banerjee Josep Lladós Saumik Bhattacharya Umapada Pal 36 5 0 01 May 2023
Do SSL Models Have Déjà Vu? A Case of Unintended Memorization in Self-supervised Learning Casey Meehan Florian Bordes Pascal Vincent Kamalika Chaudhuri Chuan Guo 36 18 0 26 Apr 2023
A Cookbook of Self-Supervised Learning Randall Balestriero Mark Ibrahim Vlad Sobal Ari S. Morcos Shashank Shekhar ... Pierre Fernandez Amir Bar Hamed Pirsiavash Yann LeCun Micah Goldblum SyDa FedML SSL 50 275 0 24 Apr 2023
The Disharmony between BN and ReLU Causes Gradient Explosion, but is Offset by the Correlation between Activations Inyoung Paik Jaesik Choi 26 0 0 23 Apr 2023
Self-supervised Learning by View Synthesis Shaoteng Liu Xiangyu Zhang T. Hu Jiaya Jia 3DV ViT 40 1 0 22 Apr 2023
Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis Yankun Wu Yuta Nakashima Noa Garcia CoGe DiffM 37 26 0 20 Apr 2023
Open-World Continual Learning: Unifying Novelty Detection and Continual Learning Gyuhak Kim Changnan Xiao Tatsuya Konishi Zixuan Ke Bin Liu CLL OODD 34 12 0 20 Apr 2023
Explaining, Analyzing, and Probing Representations of Self-Supervised Learning Models for Sensor-based Human Activity Recognition Bulat Khaertdinov S. Asteriadis 29 3 0 14 Apr 2023
A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation Florian Bordes Samuel Lavoie Randall Balestriero Nicolas Ballas Pascal Vincent SSL 40 5 0 11 Apr 2023
EMP-SSL: Towards Self-Supervised Learning in One Training Epoch Shengbang Tong Yubei Chen Yi Ma Yann LeCun 26 24 0 08 Apr 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 30 41 0 07 Apr 2023
Multi-Level Contrastive Learning for Dense Prediction Task Qiushan Guo Yizhou Yu Yi-Xin Jiang Jiannan Wu Zehuan Yuan Ping Luo SSL 32 2 0 04 Apr 2023
Anatomically aware dual-hop learning for pulmonary embolism detection in CT pulmonary angiograms Florin Condrea S. Rapaka Lucian Itu Puneet Sharma J. Sperl Mohamed Ali Marius Leordeanu 37 5 0 30 Mar 2023
Kaizen: Practical Self-supervised Continual Learning with Continual Fine-tuning Chi Ian Tang Lorena Qendro Dimitris Spathis F. Kawsar Cecilia Mascolo Akhil Mathur CLL 27 11 0 30 Mar 2023