mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization

mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization

19 February 2023

Papers citing "mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization"

15 / 15 papers shown

Title
Combinatorial Multi-armed Bandits: Arm Selection via Group Testing Arpan Mukherjee Shashanka Ubaru K. Murugesan Karthikeyan Shanmugam A. Tajer 37 1 0 14 Oct 2024
Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Training Zhanpeng Zhou Mingze Wang Yuchen Mao Bingrui Li Junchi Yan AAML 62 0 0 14 Oct 2024
Improving SAM Requires Rethinking its Optimization Formulation Wanyun Xie Fabian Latorre Kimon Antonakopoulos Thomas Pethick V. Cevher 34 1 0 17 Jul 2024
A Precise Characterization of SGD Stability Using Loss Surface Geometry Gregory Dexter Borja Ocejo S. Keerthi Aman Gupta Ayan Acharya Rajiv Khanna MLT 20 0 0 22 Jan 2024
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 44 1 0 29 Nov 2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima Dongkuk Si Chulhee Yun 28 15 0 16 Jun 2023
Normalization Layers Are All That Sharpness-Aware Minimization Needs Maximilian Mueller Tiffany J. Vlaar David Rolnick Matthias Hein 19 18 0 07 Jun 2023
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima Peter L. Bartlett Philip M. Long Olivier Bousquet 70 34 0 04 Oct 2022
Understanding Gradient Descent on Edge of Stability in Deep Learning Sanjeev Arora Zhiyuan Li A. Panigrahi MLT 80 89 0 19 May 2022
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 121 98 0 16 Oct 2021
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 88 98 0 13 Oct 2021
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 86 72 0 29 Sep 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 281 2,888 0 15 Sep 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 296 39,194 0 01 Sep 2014