Large Batch Training of Convolutional Networks

13 August 2017

Yang You

Igor Gitman

Boris Ginsburg

ODL

ArXiv PDF HTML

Papers citing "Large Batch Training of Convolutional Networks"

50 / 545 papers shown

Title
Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning Chongjian Ge Jiangliu Wang Zhan Tong Shoufa Chen Yibing Song Ping Luo SSL 22 27 0 30 Mar 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 37 30 0 27 Mar 2023
Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition Jun Cen Shiwei Zhang Xiang Wang Yixuan Pei Zhiwu Qing Yingya Zhang Qifeng Chen 44 3 0 25 Mar 2023
Mathematical Challenges in Deep Learning V. Nia Guojun Zhang I. Kobyzev Michael R. Metel Xinlin Li ... S. Hemati M. Asgharian Linglong Kong Wulong Liu Boxing Chen AI4CE VLM 37 1 0 24 Mar 2023
All4One: Symbiotic Neighbour Contrastive Learning via Self-Attention and Redundancy Reduction Imanol G. Estepa Ignacio Sarasúa Bhalaji Nagarajan Petia Radeva SSL 48 8 0 16 Mar 2023
Unsupervised Facial Expression Representation Learning with Contrastive Local Warping Fanglei Xue Yifan Sun Yi Yang CVBM 41 6 0 16 Mar 2023
Enhancing Unsupervised Audio Representation Learning via Adversarial Sample Generation Yulin Pan Xiangteng He Biao Gong Yuxin Peng Yiliang Lv SSL 24 0 0 15 Mar 2023
Stabilizing Transformer Training by Preventing Attention Entropy Collapse Shuangfei Zhai Tatiana Likhomanenko Etai Littwin Dan Busbridge Jason Ramapuram Yizhe Zhang Jiatao Gu J. Susskind AAML 48 68 0 11 Mar 2023
Self-Supervised One-Shot Learning for Automatic Segmentation of StyleGAN Images Ankit V. Manerikar A. Kak VLM 29 2 0 10 Mar 2023
Distortion-Disentangled Contrastive Learning Jinfeng Wang Sifan Song Jionglong Su S. Kevin Zhou SSL 41 4 0 09 Mar 2023
InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning Ziheng Qin Kaidi Wang Zangwei Zheng Jianyang Gu Xiang Peng ... Daquan Zhou Lei Shang Baigui Sun Xuansong Xie Yang You 127 47 0 08 Mar 2023
MAST: Masked Augmentation Subspace Training for Generalizable Self-Supervised Priors Chen Huang Hanlin Goh Jiatao Gu J. Susskind SSL OOD 60 6 0 07 Mar 2023
PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling Yuan Liu Songyang Zhang Jiacheng Chen Kai-xiang Chen Dahua Lin 75 28 0 04 Mar 2023
What Is Missing in IRM Training and Evaluation? Challenges and Solutions Yihua Zhang Pranay Sharma Parikshit Ram Min-Fong Hong Kush R. Varshney Sijia Liu 34 11 0 04 Mar 2023
Towards Democratizing Joint-Embedding Self-Supervised Learning Florian Bordes Randall Balestriero Pascal Vincent 30 20 0 03 Mar 2023
Dissolving Is Amplifying: Towards Fine-Grained Anomaly Detection Jian Shi Pengyi Zhang Ni Zhang Hakim Ghazzai Y. Massoud MedIm 43 6 0 28 Feb 2023
RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data Sangwoo Mo Jong-Chyi Su Chih-Yao Ma Mido Assran Ishan Misra Licheng Yu Sean Bell AI4TS 29 13 0 28 Feb 2023
Efficient Masked Autoencoders with Self-Consistency Zhaowen Li Yousong Zhu Zhiyang Chen Wei Li Chaoyang Zhao Rui Zhao Ming Tang Jinqiao Wang 53 2 0 28 Feb 2023
Internet Explorer: Targeted Representation Learning on the Open Web Alexander C. Li Ellis L Brown Alexei A. Efros Deepak Pathak VLM 24 24 0 27 Feb 2023
The Edge of Orthogonality: A Simple View of What Makes BYOL Tick Pierre Harvey Richemond Allison C. Tam Yunhao Tang Florian Strub Bilal Piot Felix Hill SSL 31 9 0 09 Feb 2023
DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule Maor Ivgi Oliver Hinder Y. Carmon ODL 28 57 0 08 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 31 47 0 02 Feb 2023
Unifying Synergies between Self-supervised Learning and Dynamic Computation Tarun Krishna Ayush K. Rai Alexandru Drimbarean Eric Arazo Paul Albert Alan F. Smeaton Kevin McGuinness Noel E. O'Connor 24 0 0 22 Jan 2023
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture Mahmoud Assran Quentin Duval Ishan Misra Piotr Bojanowski Pascal Vincent Michael G. Rabbat Yann LeCun Nicolas Ballas SSL AI4TS MDE 35 326 0 19 Jan 2023
SemPPL: Predicting pseudo-labels for better contrastive representations Matko Bovsnjak Pierre Harvey Richemond Nenad Tomašev Florian Strub Jacob Walker Felix Hill Lars Buesing Razvan Pascanu Charles Blundell Jovana Mitrović SSL VLM 49 9 0 12 Jan 2023
Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning Jidong Ge YuXiang Liu Jie Gui Lanting Fang Ming Lin James T. Kwok LiGuo Huang B. Luo SSL 22 5 0 08 Jan 2023
Learning by Sorting: Self-supervised Learning with Group Ordering Constraints Nina Shvetsova Felix Petersen Anna Kukleva Bernt Schiele Hilde Kuehne SSL 49 12 0 05 Jan 2023
Unsupervised Manifold Linearizing and Clustering Tianjiao Ding Shengbang Tong Kwan Ho Ryan Chan Xili Dai Yi Ma B. Haeffele 30 11 0 04 Jan 2023
Learning Decorrelated Representations Efficiently Using Fast Fourier Transform Yutaro Shigeto Masashi Shimbo Yuya Yoshikawa A. Takeuchi 27 0 0 04 Jan 2023
Disjoint Masking with Joint Distillation for Efficient Masked Image Modeling Xin Ma Chang-Shu Liu Chunyu Xie Long Ye Yafeng Deng Xiang Ji 30 9 0 31 Dec 2022
Offline Clustering Approach to Self-supervised Learning for Class-imbalanced Image Data H. Chang Sungkyun Chang SSL 11 0 0 22 Dec 2022
Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning J. Denize Jaonary Rabarisoa Astrid Orcesi Romain Hérault SSL 19 6 0 21 Dec 2022
MoQuad: Motion-focused Quadruple Construction for Video Contrastive Learning Yuan Liu Jiacheng Chen Hao Wu 38 2 0 21 Dec 2022
Input Normalized Stochastic Gradient Descent Training of Deep Neural Networks S. Atici Hongyi Pan Ahmet Enis Cetin ODL 11 0 0 20 Dec 2022
A soft nearest-neighbor framework for continual semi-supervised learning Zhiqi Kang Enrico Fini Moin Nabi Elisa Ricci Alahari Karteek SSL BDL CLL 36 17 0 09 Dec 2022
Benchmarking Self-Supervised Learning on Diverse Pathology Datasets Mingu Kang Heon Song Seonwook Park Donggeun Yoo Sérgio Pereira 13 129 0 09 Dec 2022
Training Data Influence Analysis and Estimation: A Survey Zayd Hammoudeh Daniel Lowd TDI 29 83 0 09 Dec 2022
Scaling Language-Image Pre-training via Masking Yanghao Li Haoqi Fan Ronghang Hu Christoph Feichtenhofer Kaiming He CLIP VLM 42 318 0 01 Dec 2022
A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix Wenbin Li Meihao Kong Xuesong Yang Lei Wang Jing Huo Yang Gao Jiebo Luo 33 0 0 26 Nov 2022
Ladder Siamese Network: a Method and Insights for Multi-level Self-Supervised Learning Ryota Yoshihashi Shuhei Nishimura Dai Yonebayashi Yuya Otsuka Tomohiro Tanaka Takashi Miyazaki SSL 24 2 0 25 Nov 2022
Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size Alexander Nikulin Vladislav Kurenkov Denis Tarasov Dmitry Akimov Sergey Kolesnikov OffRL 36 14 0 20 Nov 2022
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 35 60 0 17 Nov 2022
How to Fine-Tune Vision Models with SGD Ananya Kumar Ruoqi Shen Sébastien Bubeck Suriya Gunasekar VLM 14 29 0 17 Nov 2022
MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis Tianhong Li Huiwen Chang Shlok Kumar Mishra Han Zhang Dina Katabi Dilip Krishnan 41 154 0 16 Nov 2022
Homomorphic Self-Supervised Learning Thomas Anderson Keller Xavier Suau Luca Zappella SSL 19 2 0 15 Nov 2022
LLEDA -- Lifelong Self-Supervised Domain Adaptation Mamatha Thota Dewei Yi Georgios Leontidis CLL 73 7 0 12 Nov 2022
Masked Contrastive Representation Learning Yuan Yao Nandakishor Desai M. Palaniswami SSL 22 8 0 11 Nov 2022
Local Manifold Augmentation for Multiview Semantic Consistency Yu Yang Wing Yin Cheung Chang-rui Liu Xiang Ji 39 1 0 05 Nov 2022
A Theoretical Study on Solving Continual Learning Gyuhak Kim Changnan Xiao Tatsuya Konishi Zixuan Ke Bin Liu CLL 27 71 0 04 Nov 2022
Unsupervised Visual Representation Learning via Mutual Information Regularized Assignment Dong Lee Sung-Ik Choi Hyunwoo J. Kim Sae-Young Chung SSL 37 7 0 04 Nov 2022