Sharpness-Aware Minimization for Efficiently Improving Generalization

3 October 2020

Papers citing "Sharpness-Aware Minimization for Efficiently Improving Generalization"

50 / 867 papers shown

Title
Lookaround Optimizer: $k$ steps around, 1 step average Jiangtao Zhang Shunyu Liu Mingli Song Tongtian Zhu Zhenxing Xu Mingli Song MoMe 37 6 0 13 Jun 2023
Gradient Ascent Post-training Enhances Language Model Generalization Dongkeun Yoon Joel Jang Sungdong Kim Minjoon Seo VLM AI4CE 26 3 0 12 Jun 2023
FasterViT: Fast Vision Transformers with Hierarchical Attention Ali Hatamizadeh Greg Heinrich Hongxu Yin Andrew Tao J. Álvarez Jan Kautz Pavlo Molchanov ViT 25 67 0 09 Jun 2023
Differentially Private Sharpness-Aware Training Jinseong Park Hoki Kim Yujin Choi Jaewook Lee 29 8 0 09 Jun 2023
Boosting Adversarial Transferability by Achieving Flat Local Maxima Zhijin Ge Hongying Liu Xiaosen Wang Fanhua Shang Yuanyuan Liu AAML 14 40 0 08 Jun 2023
Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations Hyeonjeong Ha Minseon Kim Sung Ju Hwang OOD AAML 30 5 0 08 Jun 2023
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning Libin Zhu Chaoyue Liu Adityanarayanan Radhakrishnan M. Belkin 32 14 0 07 Jun 2023
Normalization Layers Are All That Sharpness-Aware Minimization Needs Maximilian Mueller Tiffany J. Vlaar David Rolnick Matthias Hein 27 18 0 07 Jun 2023
Optimal Transport Model Distributional Robustness Van-Anh Nguyen Trung Le Anh Tuan Bui Thanh-Toan Do Dinh Q. Phung OOD 30 3 0 07 Jun 2023
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent Tongtian Zhu Fengxiang He Kaixuan Chen Mingli Song Dacheng Tao 34 15 0 05 Jun 2023
ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction Jia Guo Shuai Lu Lize Jia Weihang Zhang Huiqi Li 24 23 0 05 Jun 2023
When Decentralized Optimization Meets Federated Learning Hongchang Gao My T. Thai Jie Wu FedML 26 22 0 05 Jun 2023
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression Runtian Zhai Bing Liu Andrej Risteski Zico Kolter Pradeep Ravikumar SSL 28 9 0 01 Jun 2023
Toward Understanding Why Adam Converges Faster Than SGD for Transformers Yan Pan Yuanzhi Li 33 41 0 31 May 2023
Inconsistency, Instability, and Generalization Gap of Deep Neural Network Training Rie Johnson Tong Zhang 11 5 0 31 May 2023
Multi-Dataset Co-Training with Sharpness-Aware Optimization for Audio Anti-spoofing Hye-jin Shim Jee-weon Jung Tomi Kinnunen 21 13 0 31 May 2023
Generalization Bounds for Magnitude-Based Pruning via Sparse Matrix Sketching E. Guha Prasanjit Dubey X. Huo MLT 34 1 0 30 May 2023
SANE: The phases of gradient descent through Sharpness Adjusted Number of Effective parameters Lawrence Wang Stephen J. Roberts 24 0 0 29 May 2023
A Three-regime Model of Network Pruning Yefan Zhou Yaoqing Yang Arin Chang Michael W. Mahoney 34 10 0 28 May 2023
The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent Lei Wu Weijie J. Su MLT 30 21 0 27 May 2023
Learning Capacity: A Measure of the Effective Dimensionality of a Model Daiwei Chen Wei-Di Chang Pratik Chaudhari 37 3 0 27 May 2023
Sharpness-Aware Minimization Leads to Low-Rank Features Maksym Andriushchenko Dara Bahri H. Mobahi Nicolas Flammarion AAML 25 25 0 25 May 2023
Two-timescale Extragradient for Finding Local Minimax Points Jiseok Chae Kyuwon Kim Donghwan Kim 10 4 0 25 May 2023
Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Term Yun Yue Jiadi Jiang Zhiling Ye Ni Gao Yongchao Liu Kecheng Zhang MLAU ODL 33 11 0 25 May 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 34 6 0 25 May 2023
The Crucial Role of Normalization in Sharpness-Aware Minimization Yan Dai Kwangjun Ahn S. Sra 21 17 0 24 May 2023
Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training Yi Shi Yingqi Liu Yan Sun Zihao Lin Li Shen Xueqian Wang Dacheng Tao FedML 45 10 0 24 May 2023
Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning Moonseok Choi Hyungi Lee G. Nam Juho Lee 40 2 0 24 May 2023
Sharpness-Aware Data Poisoning Attack Pengfei He Han Xu J. Ren Yingqian Cui Hui Liu Charu C. Aggarwal Jiliang Tang AAML 47 7 0 24 May 2023
On progressive sharpening, flat minima and generalisation L. MacDonald Jack Valmadre Simon Lucey 27 4 0 24 May 2023
Tied-Augment: Controlling Representation Similarity Improves Data Augmentation Emirhan Kurtuluş Zichao Li Yann N. Dauphin E. D. Cubuk 38 3 0 22 May 2023
Improving Convergence and Generalization Using Parameter Symmetries Bo Zhao Robert Mansel Gower Robin Walters Rose Yu MoMe 33 13 0 22 May 2023
Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization Z. Fu Yixuan Su Zaiqiao Meng Nigel Collier MedIm 32 3 0 22 May 2023
Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond Itai Kreisler Mor Shpigel Nacson Daniel Soudry Y. Carmon 33 13 0 22 May 2023
Loss Spike in Training Neural Networks Zhongwang Zhang Z. Xu 36 4 0 20 May 2023
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape Yan Sun Li Shen Shi-Yong Chen Liang Ding Dacheng Tao FedML 37 33 0 19 May 2023
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency Lingfeng Shen Weiting Tan Boyuan Zheng Daniel Khashabi VLM 39 6 0 18 May 2023
Sharpness & Shift-Aware Self-Supervised Learning Ngoc N. Tran S. Duong Hoang Phan Tung Pham Dinh Q. Phung Trung Le SSL 40 1 0 17 May 2023
DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning Jun Shu Xiang Yuan Deyu Meng Zongben Xu 28 4 0 13 May 2023
Understanding and Improving Model Averaging in Federated Learning on Heterogeneous Data Tailin Zhou Zehong Lin Jinchao Zhang Danny H. K. Tsang MoMe FedML 38 12 0 13 May 2023
A Survey on the Robustness of Computer Vision Models against Common Corruptions Shunxin Wang Raymond N. J. Veldhuis Christoph Brune N. Strisciuglio OOD VLM 30 11 0 10 May 2023
Sharpness-Aware Minimization Alone can Improve Adversarial Robustness Zeming Wei Jingyu Zhu Yihao Zhang AAML 32 10 0 09 May 2023
CrAFT: Compression-Aware Fine-Tuning for Efficient Visual Task Adaptation J. Heo S. Azizi A. Fayyazi Massoud Pedram 28 0 0 08 May 2023
LOGO-Former: Local-Global Spatio-Temporal Transformer for Dynamic Facial Expression Recognition Fuyan Ma Bin Sun Shutao Li ViT 27 20 0 05 May 2023
Venn Diagram Multi-label Class Interpretation of Diabetic Foot Ulcer with Color and Sharpness Enhancement M. Hasan Moi Hoon Yap M. Hasan 18 2 0 01 May 2023
Model-agnostic Measure of Generalization Difficulty Akhilan Boopathy Kevin Liu Jaedong Hwang Shu Ge Asaad Mohammedsaleh Ila Fiete 80 4 0 01 May 2023
Early Detection of Alzheimer's Disease using Bottleneck Transformers Arunima Jaiswal Ananya Sadana MedIm 26 2 0 01 May 2023
Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy Yi Shi Kang Wei Li Shen Yingqi Liu Xueqian Wang Bo Yuan Dacheng Tao FedML 41 2 0 01 May 2023
An Adaptive Policy to Employ Sharpness-Aware Minimization Weisen Jiang Hansi Yang Yu Zhang James T. Kwok AAML 83 31 0 28 Apr 2023
Self-discipline on multiple channels Jiutian Zhao Liangchen Luo Hao Wang 32 0 0 27 Apr 2023