ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks

23 February 2021

Papers citing "ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks"

50 / 193 papers shown

Title
Promoting Exploration in Memory-Augmented Adam using Critical Momenta Pranshu Malviya Gonçalo Mordido A. Baratin Reza Babanezhad Harikandeh Jerry Huang Simon Lacoste-Julien Razvan Pascanu Sarath Chandar ODL 33 1 0 18 Jul 2023
Sharpness-Aware Graph Collaborative Filtering Huiyuan Chen Chin-Chia Michael Yeh Yujie Fan Yan Zheng Junpeng Wang Vivian Lai Mahashweta Das Hao Yang 31 5 0 18 Jul 2023
FAM: Relative Flatness Aware Minimization Linara Adilova Amr Abourayya Jianning Li Amin Dada Henning Petzka Jan Egger Jens Kleesiek Michael Kamp ODL 29 1 0 05 Jul 2023
Systematic Investigation of Sparse Perturbed Sharpness-Aware Minimization Optimizer Peng Mi Li Shen Tianhe Ren Yiyi Zhou Tianshuo Xu Xiaoshuai Sun Tongliang Liu Rongrong Ji Dacheng Tao AAML 37 2 0 30 Jun 2023
Adaptive Sharpness-Aware Pruning for Robust Sparse Networks Anna Bair Hongxu Yin Maying Shen Pavlo Molchanov J. Álvarez 40 10 0 25 Jun 2023
The Inductive Bias of Flatness Regularization for Deep Matrix Factorization Khashayar Gatmiry Zhiyuan Li Ching-Yao Chuang Sashank J. Reddi Tengyu Ma Stefanie Jegelka ODL 25 11 0 22 Jun 2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima Dongkuk Si Chulhee Yun 28 15 0 16 Jun 2023
The Split Matters: Flat Minima Methods for Improving the Performance of GNNs N. Lell A. Scherp 43 1 0 15 Jun 2023
Tokenization with Factorized Subword Encoding David Samuel Lilja Øvrelid 41 1 0 13 Jun 2023
Normalization Layers Are All That Sharpness-Aware Minimization Needs Maximilian Mueller Tiffany J. Vlaar David Rolnick Matthias Hein 27 18 0 07 Jun 2023
Optimal Transport Model Distributional Robustness Van-Anh Nguyen Trung Le Anh Tuan Bui Thanh-Toan Do Dinh Q. Phung OOD 30 3 0 07 Jun 2023
Multi-Dataset Co-Training with Sharpness-Aware Optimization for Audio Anti-spoofing Hye-jin Shim Jee-weon Jung Tomi Kinnunen 21 13 0 31 May 2023
Sharpness-Aware Minimization Leads to Low-Rank Features Maksym Andriushchenko Dara Bahri H. Mobahi Nicolas Flammarion AAML 25 25 0 25 May 2023
Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Term Yun Yue Jiadi Jiang Zhiling Ye Ni Gao Yongchao Liu Kecheng Zhang MLAU ODL 30 11 0 25 May 2023
The Crucial Role of Normalization in Sharpness-Aware Minimization Yan Dai Kwangjun Ahn S. Sra 21 17 0 24 May 2023
Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training Yi Shi Yingqi Liu Yan Sun Zihao Lin Li Shen Xueqian Wang Dacheng Tao FedML 45 10 0 24 May 2023
Improving Convergence and Generalization Using Parameter Symmetries Bo Zhao Robert Mansel Gower Robin Walters Rose Yu MoMe 33 13 0 22 May 2023
Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination Ming Hu Zhihao Yue Zhiwei Ling Cheng Chen Yihao Huang Xian Wei Xiang Lian Yang Liu Mingsong Chen FedML 19 8 0 18 May 2023
Sharpness & Shift-Aware Self-Supervised Learning Ngoc N. Tran S. Duong Hoang Phan Tung Pham Dinh Q. Phung Trung Le SSL 34 1 0 17 May 2023
CrAFT: Compression-Aware Fine-Tuning for Efficient Visual Task Adaptation J. Heo S. Azizi A. Fayyazi Massoud Pedram 28 0 0 08 May 2023
Venn Diagram Multi-label Class Interpretation of Diabetic Foot Ulcer with Color and Sharpness Enhancement M. Hasan Moi Hoon Yap M. Hasan 18 2 0 01 May 2023
Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy Yi Shi Kang Wei Li Shen Yingqi Liu Xueqian Wang Bo Yuan Dacheng Tao FedML 38 2 0 01 May 2023
An Adaptive Policy to Employ Sharpness-Aware Minimization Weisen Jiang Hansi Yang Yu Zhang James T. Kwok AAML 83 31 0 28 Apr 2023
Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimization Mingli Zhu Shaokui Wei Li Shen Yanbo Fan Baoyuan Wu AAML 36 51 0 24 Apr 2023
Going Further: Flatness at the Rescue of Early Stopping for Adversarial Example Transferability Martin Gubri Maxime Cordy Yves Le Traon AAML 20 3 1 05 Apr 2023
Sample4Geo: Hard Negative Sampling For Cross-View Geo-Localisation Fabian Deuser Konrad Habel Norbert Oswald 25 54 0 21 Mar 2023
Make Landscape Flatter in Differentially Private Federated Learning Yi Shi Yingqi Liu Kang Wei Li Shen Xueqian Wang Dacheng Tao FedML 25 54 0 20 Mar 2023
Rethinking Model Ensemble in Transfer-based Adversarial Attacks Huanran Chen Yichi Zhang Yinpeng Dong Xiao Yang Hang Su Junyi Zhu AAML 28 56 0 16 Mar 2023
Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization Xingxuan Zhang Renzhe Xu Han Yu Hao Zou Peng Cui 16 39 0 03 Mar 2023
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks Hao Sun Li Shen Qihuang Zhong Liang Ding Shi-Yong Chen Jingwei Sun Jing Li Guangzhong Sun Dacheng Tao 49 31 0 01 Mar 2023
Towards Stable Test-Time Adaptation in Dynamic Wild World Shuaicheng Niu Jiaxiang Wu Yifan Zhang Z. Wen Yaofo Chen P. Zhao Mingkui Tan TTA 35 248 0 24 Feb 2023
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta S. Keerthi Ayan Acharya Borja Ocejo Gregory Dexter Rajiv Khanna D. Durfee Rahul Mazumder AAML 18 7 0 19 Feb 2023
Improving Differentiable Architecture Search via Self-Distillation Xunyu Zhu Jian Li Yong Liu Weiping Wang 24 7 0 11 Feb 2023
Flat Seeking Bayesian Neural Networks Van-Anh Nguyen L. Vuong Hoang Phan Thanh-Toan Do Dinh Q. Phung Trung Le BDL 33 8 0 06 Feb 2023
Exploring the Effect of Multi-step Ascent in Sharpness-Aware Minimization Hoki Kim Jinseong Park Yujin Choi Woojin Lee Jaewook Lee 20 9 0 27 Jan 2023
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurelien Lucchi 23 13 0 19 Jan 2023
Stability Analysis of Sharpness-Aware Minimization Hoki Kim Jinseong Park Yujin Choi Jaewook Lee 39 12 0 16 Jan 2023
GoogLe2Net: Going Transverse with Convolutions Yuanpeng He 22 2 0 01 Jan 2023
A Generalization of ViT/MLP-Mixer to Graphs Xiaoxin He Bryan Hooi T. Laurent Adam Perold Yann LeCun Xavier Bresson 47 88 0 27 Dec 2022
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging Peng Lu I. Kobyzev Mehdi Rezagholizadeh Ahmad Rashid A. Ghodsi Philippe Langlais MoMe 41 11 0 12 Dec 2022
Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks Yihan Wu Aleksandar Bojchevski Heng Huang AAML 34 30 0 09 Dec 2022
Improving Multi-task Learning via Seeking Task-based Flat Regions Hoang Phan Lam C. Tran Ngoc N. Tran Nhat Ho Dinh Q. Phung Trung Le 27 11 0 24 Nov 2022
Efficient Generalization Improvement Guided by Random Weight Perturbation Tao Li Wei Yan Zehao Lei Yingwen Wu Kun Fang Ming Yang X. Huang AAML 40 6 0 21 Nov 2022
SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness Gonçalo Mordido Sébastien Henwood Sarath Chandar Franccois Leduc-Primeau AAML 30 0 0 18 Nov 2022
How Does Sharpness-Aware Minimization Minimize Sharpness? Kaiyue Wen Tengyu Ma Zhiyuan Li AAML 23 47 0 10 Nov 2022
Learning Cross-view Geo-localization Embeddings via Dynamic Weighted Decorrelation Regularization Ting Wang Zhedong Zheng Zunjie Zhu Yuhan Gao Yi Yang Chenggang Yan 25 35 0 10 Nov 2022
Sufficient Invariant Learning for Distribution Shift Taero Kim Sungjun Lim Kyungwoo Song OOD 31 2 0 24 Oct 2022
K-SAM: Sharpness-Aware Minimization at the Speed of SGD Renkun Ni Ping Yeh-Chiang Jonas Geiping Micah Goldblum A. Wilson Tom Goldstein 26 8 0 23 Oct 2022
Rethinking Sharpness-Aware Minimization as Variational Inference Szilvia Ujváry Zsigmond Telek A. Kerekes Anna Mészáros Ferenc Huszár 33 8 0 19 Oct 2022
GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization Zhiyuan Zhang Ruixuan Luo Qi Su Xueting Sun 29 11 0 13 Oct 2022