Large Batch Training of Convolutional Networks

13 August 2017

Yang You

Igor Gitman

Boris Ginsburg

ODL

ArXiv PDF HTML

Papers citing "Large Batch Training of Convolutional Networks"

50 / 544 papers shown

Title
AlphaGrad: Non-Linear Gradient Normalization Optimizer Soham Sane ODL 56 0 0 22 Apr 2025
Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Training Yi Hu Jinhang Zuo Eddie Zhang Bob Iannucci Carlee Joe-Wong 37 0 0 13 Apr 2025
Beyond Accuracy: What Matters in Designing Well-Behaved Models? Robin Hesse Doğukan Bağcı Bernt Schiele Simone Schaub-Meyer Stefan Roth VLM 62 0 0 21 Mar 2025
Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learning Jisoo Kim Sungmin Kang Sunwoo Lee FedML 52 0 0 14 Mar 2025
Structured Preconditioners in Adaptive Optimization: A Unified Analysis Shuo Xie Tianhao Wang Sashank J. Reddi Sanjiv Kumar Zhiyuan Li 45 1 0 13 Mar 2025
Implicit Contrastive Representation Learning with Guided Stop-gradient Byeongchan Lee Sehyun Lee SSL 89 2 0 12 Mar 2025
USP: Unified Self-Supervised Pretraining for Image Generation and Understanding Xiangxiang Chu Renda Li Yong Wang 65 0 0 08 Mar 2025
Spatial Context-Driven Positive Pair Sampling for Enhanced Histopathology Image Classification Willmer Rafell Quinones Robles Sakonporn Noree Y. Ko Bryan Wong JongWoo Kim Mun Yi 47 0 0 07 Mar 2025
Super-Resolution for Interferometric Imaging: Model Comparisons and Performance Analysis Hasan Berkay Abdioglu Rana Gursoy Yagmur Isik Ibrahim Cem Balci Taha Unal ... Mustafa Ismail Inal Nehir Serin Muhammed Furkan Kosar G. B. Esmer H. Uvet 64 0 0 24 Feb 2025
Discovery and Deployment of Emergent Robot Swarm Behaviors via Representation Learning and Real2Sim2Real Transfer Connor Mattson Varun Raveendra Ricardo Vega Cameron Nowzari Daniel S. Drew Daniel S. Brown 50 0 0 21 Feb 2025
Why does my medical AI look at pictures of birds? Exploring the efficacy of transfer learning across domain boundaries F. Jonske M. Kim Enrico Nasca J. Evers Johannes Haubold ... F. Nensa Michael Kamp C. Seibold Jan Egger Jens Kleesiek 79 1 0 17 Feb 2025
Vision-Language Models for Edge Networks: A Comprehensive Survey Ahmed Sharshar Latif U. Khan Waseem Ullah Mohsen Guizani VLM 70 3 0 11 Feb 2025
Gradient Multi-Normalization for Stateless and Scalable LLM Training M. Scetbon Chao Ma Wenbo Gong Edward Meeds 99 1 0 10 Feb 2025
Nearly Lossless Adaptive Bit Switching Haiduo Huang Zhenhua Liu Tian Xia Wenzhe zhao Pengju Ren MQ 63 0 0 03 Feb 2025
Learning Versatile Optimizers on a Compute Diet A. Moudgil Boris Knyazev Guillaume Lajoie Eugene Belilovsky 168 0 0 22 Jan 2025
Memory Storyboard: Leveraging Temporal Segmentation for Streaming Self-Supervised Learning from Egocentric Videos Yanlai Yang Mengye Ren 201 0 0 21 Jan 2025
A Hessian-informed hyperparameter optimization for differential learning rate Shiyun Xu Zhiqi Bu Yiliang Zhang Ian Barnett 39 1 0 12 Jan 2025
Gaussian Masked Autoencoders Jathushan Rajasegaran Xinlei Chen Rulilong Li Christoph Feichtenhofer Jitendra Malik Shiry Ginosar 3DGS 45 1 0 06 Jan 2025
PiLaMIM: Toward Richer Visual Representations by Integrating Pixel and Latent Masked Image Modeling Junmyeong Lee Eui Jun Hwang Sukmin Cho Jong C. Park 40 0 0 06 Jan 2025
Temporal Context Consistency Above All: Enhancing Long-Term Anticipation by Learning and Enforcing Temporal Constraints Alberto Maté Mariella Dimiccoli AI4TS 31 0 0 27 Dec 2024
Asymmetric Learning for Spectral Graph Neural Networks Fangbing Liu Qing Wang 88 0 0 16 Dec 2024
ParaGAN: A Scalable Distributed Training Framework for Generative Adversarial Networks Ziji Shi Jialin Li Yang You 26 1 0 06 Nov 2024
Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models Junjiao Tian Chengyue Huang Z. Kira 44 1 0 03 Nov 2024
Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training Atli Kosson Bettina Messmer Martin Jaggi AI4CE 22 2 0 31 Oct 2024
How Does Critical Batch Size Scale in Pre-training? Hanlin Zhang Depen Morwani Nikhil Vyas Jingfeng Wu Difan Zou Udaya Ghai Dean Phillips Foster Sham Kakade 80 8 0 29 Oct 2024
LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization Jui-Nan Yen Si Si Zhao Meng Felix X. Yu Sai Surya Duvvuri Inderjit Dhillon Cho-Jui Hsieh Sanjiv Kumar 27 3 0 27 Oct 2024
OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagery P. Dias A. Tsaris Jordan Bowman Abhishek Potnis Jacob Arndt H. Yang D. Lunga 29 5 0 25 Oct 2024
Rethinking Positive Pairs in Contrastive Learning Jiantao Wu Shentong Mo Zhenhua Feng Sara Atito Josef Kitler Muhammad Awais SSL VLM 48 3 0 23 Oct 2024
SigCLR: Sigmoid Contrastive Learning of Visual Representations Ömer Veysel Çağatan 24 0 0 22 Oct 2024
Feature Augmentation for Self-supervised Contrastive Learning: A Closer Look Yong Zhang Rui Zhu Shifeng Zhang Xu Zhou Shifeng Chen Xiaofan Chen SSL 45 0 0 16 Oct 2024
Adaptive Diffusion Terrain Generator for Autonomous Uneven Terrain Navigation Youwei Yu Junhong Xu Lantao Liu 39 0 0 14 Oct 2024
Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax I. Butakov Alexander Sememenko Alexander Tolmachev Andrey Gladkov Marina Munkhoeva Alexey Frolov 37 0 0 09 Oct 2024
OD-Stega: LLM-Based Near-Imperceptible Steganography via Optimized Distributions Yu-Shin Huang Peter Just Krishna Narayanan Chao Tian 34 4 0 06 Oct 2024
BiSSL: A Bilevel Optimization Framework for Enhancing the Alignment Between Self-Supervised Pre-Training and Downstream Fine-Tuning Gustav Wagner Zakarias Lars Kai Hansen Zheng-Hua Tan 36 0 0 03 Oct 2024
Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes Nikita Kiselev Andrey Grabovoy 54 1 0 18 Sep 2024
EEG-Language Modeling for Pathology Detection Sam Gijsen Kerstin Ritter 47 0 0 02 Sep 2024
EMP: Enhance Memory in Data Pruning Jinying Xiao Ping Li Jie Nie Zhe Tang VLM 57 0 0 28 Aug 2024
Universal Novelty Detection Through Adaptive Contrastive Learning Hossein Mirzaei Mojtaba Nafez Mohammad Jafari Mohammad Bagher Soltani Mohammad Azizmalayeri Jafar Habibi Mohammad Sabokrou M. Rohban 32 4 0 20 Aug 2024
CoBooM: Codebook Guided Bootstrapping for Medical Image Representation Learning Azad Singh Deepak Mishra SSL 44 1 0 08 Aug 2024
Masked Angle-Aware Autoencoder for Remote Sensing Images Zhihao Li B. Hou Siteng Ma Zitong Wu Xianpeng Guo Bo Ren Licheng Jiao 49 11 0 04 Aug 2024
Towards Latent Masked Image Modeling for Self-Supervised Visual Representation Learning Yibing Wei Abhinav Gupta Pedro Morgado SSL 47 7 0 22 Jul 2024
Predicting the Best of N Visual Trackers B. Alawode S. Javed Arif Mahmood Jirí Matas 49 1 0 22 Jul 2024
Self-Supervised Video Representation Learning in a Heuristic Decoupled Perspective Changwen Zheng Wenwen Qiang Jianqi Zhang Changwen Zheng Jingyao Wang SSL 66 0 0 19 Jul 2024
OvSW: Overcoming Silent Weights for Accurate Binary Neural Networks Jingyang Xiang Zuohui Chen Siqi Li Qing Wu Yong-Jin Liu 28 1 0 07 Jul 2024
Large Batch Analysis for Adagrad Under Anisotropic Smoothness Yuxing Liu Rui Pan Tong Zhang 26 5 0 21 Jun 2024
Towards evolution of Deep Neural Networks through contrastive Self-Supervised learning Adriano Vinhas João Correia Penousal Machado SSL 29 1 0 20 Jun 2024
Mixing Natural and Synthetic Images for Robust Self-Supervised Representations Reza Akbarian Bafghi Nidhin Harilal C. Monteleoni M. Raissi DiffM 41 0 0 18 Jun 2024
Bioptic -- A Target-Agnostic Potency-Based Small Molecules Search Engine Vlad Vinogradov Ivan Izmailov Simon Steshin Kong T. Nguyen 26 0 0 13 Jun 2024
DDA: Dimensionality Driven Augmentation Search for Contrastive Learning in Laparoscopic Surgery Yuning Zhou H. Badgery Matthew Read James Bailey Catherine E. Davey 45 1 0 03 Jun 2024
AdaFisher: Adaptive Second Order Optimization via Fisher Information Damien Martins Gomes Yanlei Zhang Eugene Belilovsky Guy Wolf Mahdi S. Hosseini ODL 76 2 0 26 May 2024