Sharpness-Aware Minimization for Efficiently Improving Generalization

3 October 2020

Papers citing "Sharpness-Aware Minimization for Efficiently Improving Generalization"

50 / 867 papers shown

Title
Advancing Vision Transformers with Group-Mix Attention Chongjian Ge Xiaohan Ding Zhan Tong Li Yuan Jiangliu Wang Yibing Song Ping Luo 112 16 0 26 Nov 2023
TEA: Test-time Energy Adaptation Yige Yuan Bingbing Xu Liang Hou Fei Sun Huawei Shen Xueqi Cheng TTA VLM 34 8 0 24 Nov 2023
Achieving Margin Maximization Exponentially Fast via Progressive Norm Rescaling Mingze Wang Zeping Min Lei Wu 33 3 0 24 Nov 2023
Video-based Sequential Bayesian Homography Estimation for Soccer Field Registration Paul J. Claasen J. P. de Villiers 17 8 0 17 Nov 2023
A2XP: Towards Private Domain Generalization Geunhyeok Yu Hyoseok Hwang 39 0 0 17 Nov 2023
I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization Mingliang Xu Jiawei Hu Mingbao Lin Yonghong Tian Rongrong Ji MQ 30 10 0 16 Nov 2023
Robust Contrastive Learning With Theory Guarantee Ngoc N. Tran Lam C. Tran Hoang Phan Anh-Vu Bui Tung Pham Toan M. Tran Dinh Q. Phung Trung Le SSL NoLa 29 0 0 16 Nov 2023
Federated Learning with Manifold Regularization and Normalized Update Reaggregation Xuming An Li Shen Han Hu Yong Luo FedML 44 4 0 10 Nov 2023
3DGAUnet: 3D generative adversarial networks with a 3D U-Net based generator to achieve the accurate and effective synthesis of clinical tumor image data for pancreatic cancer Yu Shi Hannah Tang M. Baine M. A. Hollingsworth Huijing Du Dandan Zheng Chi Zhang Hongfeng Yu MedIm 11 5 0 09 Nov 2023
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications Huayi Tang Yong Liu 62 1 0 08 Nov 2023
Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization Elan Rosenfeld Andrej Risteski 25 10 0 07 Nov 2023
OmniVec: Learning robust representations with cross modal sharing Siddharth Srivastava Gaurav Sharma SSL 29 64 0 07 Nov 2023
Signal Processing Meets SGD: From Momentum to Filter Zhipeng Yao Guisong Chang Jiaqi Zhang Qi Zhang Dazhou Li Yu Zhang ODL 31 0 0 06 Nov 2023
Fully Quantized Always-on Face Detector Considering Mobile Image Sensors Haechang Lee Wongi Jeong Dongil Ryu Hyunwoo Je Albert No Kijeong Kim Se Young Chun CVBM 31 0 0 02 Nov 2023
In Search of Lost Online Test-time Adaptation: A Survey Zixin Wang Yadan Luo Liang Zheng Zhuoxiao Chen Sen Wang Zi Huang 32 15 0 31 Oct 2023
Seeking Flat Minima with Mean Teacher on Semi- and Weakly-Supervised Domain Generalization for Object Detection Ryosuke Furuta Yoichi Sato 34 0 0 30 Oct 2023
PAC-tuning:Fine-tuning Pretrained Language Models with PAC-driven Perturbed Gradient Descent Guang-Da Liu Zhiyu Xue Xitong Zhang K. Johnson Rongrong Wang 25 5 0 26 Oct 2023
ConvNets Match Vision Transformers at Scale Samuel L. Smith Andrew Brock Leonard Berrada Soham De 13 23 0 25 Oct 2023
FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning Zhuo Huang Li Shen Jun-chen Yu Bo Han Tongliang Liu FedML 29 21 0 25 Oct 2023
Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization Zhuo Huang Muyang Li Li Shen Jun-chen Yu Chen Gong Bo Han Tongliang Liu OOD 46 8 0 25 Oct 2023
Irreducible Curriculum for Language Model Pretraining Simin Fan Martin Jaggi 27 9 0 23 Oct 2023
Learning spatio-temporal patterns with Neural Cellular Automata Alex D. Richardson Tibor Antal Richard A. Blythe Linus J. Schumacher AI4CE 11 2 0 23 Oct 2023
A Quadratic Synchronization Rule for Distributed Deep Learning Xinran Gu Kaifeng Lyu Sanjeev Arora Jingzhao Zhang Longbo Huang 54 1 0 22 Oct 2023
Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolation Jianing Zhu Geng Yu Jiangchao Yao Tongliang Liu Gang Niu Masashi Sugiyama Bo Han OODD 34 30 0 21 Oct 2023
Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models Miaoxi Zhu Qihuang Zhong Li Shen Liang Ding Juhua Liu Bo Du Dacheng Tao MQ VLM 29 1 0 20 Oct 2023
Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters Gyuseong Lee Wooseok Jang Jin Hyeon Kim Jaewoo Jung Seungryong Kim MoE OOD 30 2 0 17 Oct 2023
SoTTA: Robust Test-Time Adaptation on Noisy Data Streams Taesik Gong Yewon Kim Taeckyung Lee Sorn Chottananurak Sung-Ju Lee TTA 37 27 0 16 Oct 2023
Why Does Sharpness-Aware Minimization Generalize Better Than SGD? Zixiang Chen Junkai Zhang Yiwen Kou Xiangning Chen Cho-Jui Hsieh Quanquan Gu 34 13 0 11 Oct 2023
Entropy-MCMC: Sampling from Flat Basins with Ease Bolian Li Ruqi Zhang 32 5 0 09 Oct 2023
Asymmetrically Decentralized Federated Learning Qinglun Li Miao Zhang Nan Yin Quanjun Yin Li Shen FedML 32 4 0 08 Oct 2023
FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysis Raman Dutt Ondrej Bohdal Sotirios A. Tsaftaris Timothy M. Hospedales 21 14 0 08 Oct 2023
A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment for Imbalanced Learning Zitai Wang Qianqian Xu Zhiyong Yang Yuan He Xiaochun Cao Qingming Huang 28 24 0 07 Oct 2023
TRAM: Bridging Trust Regions and Sharpness Aware Minimization Tom Sherborne Naomi Saphra Pradeep Dasigi Hao Peng 32 4 0 05 Oct 2023
SHOT: Suppressing the Hessian along the Optimization Trajectory for Gradient-Based Meta-Learning Junhoo Lee Jayeon Yoo Nojun Kwak 27 2 0 04 Oct 2023
Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learning Debora Caldarola Barbara Caputo Marco Ciccone FedML 13 7 0 02 Oct 2023
On Memorization and Privacy Risks of Sharpness Aware Minimization Young In Kim Pratiksha Agrawal J. Royset Rajiv Khanna FedML 28 3 0 30 Sep 2023
RSAM: Learning on manifolds with Riemannian Sharpness-aware Minimization Kenneth Allen Hoang-Phi Nguyen Tung Pham Ming-Jun Lai Mehrtash Harandi Dinh Q. Phung Trung Le AAML 40 3 0 29 Sep 2023
A Primer on Bayesian Neural Networks: Review and Debates Federico Danieli Konstantinos Pitas M. Vladimirova Vincent Fortuin BDL AAML 56 18 0 28 Sep 2023
Enhancing Sharpness-Aware Optimization Through Variance Suppression Bingcong Li G. Giannakis AAML 28 19 0 27 Sep 2023
Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR) Guo-qing Jiang Jinlong Liu Zixiang Ding Lin Guo W. Lin AI4CE 24 1 0 24 Sep 2023
Probabilistic Weight Fixing: Large-scale training of neural network weight uncertainties for quantization Christopher Subia-Waud S. Dasmahapatra UQCV MQ 21 0 0 24 Sep 2023
Domain-Guided Conditional Diffusion Model for Unsupervised Domain Adaptation Yulong Zhang Shu Han Chen Weisen Jiang Yu Zhang Jiangang Lu James T. Kwok DiffM 49 5 0 23 Sep 2023
Sharpness-Aware Minimization and the Edge of Stability Philip M. Long Peter L. Bartlett AAML 27 9 0 21 Sep 2023
Create and Find Flatness: Building Flat Training Spaces in Advance for Continual Learning Wenhang Shi Yiren Chen Zhe Zhao Wei Lu Kimmo Yan Xiaoyong Du CLL 35 5 0 20 Sep 2023
Automatic Bat Call Classification using Transformer Networks Frank Fundel Daniel Braun Sebastian Gottwald 14 6 0 20 Sep 2023
EPTQ: Enhanced Post-Training Quantization via Label-Free Hessian Ofir Gordon H. Habi Arnon Netzer MQ 41 1 0 20 Sep 2023
Gradient constrained sharpness-aware prompt learning for vision-language models Liangchen Liu Nannan Wang Dawei Zhou Xinbo Gao Decheng Liu Xi Yang Tongliang Liu VLM 33 2 0 14 Sep 2023
Exploring Flat Minima for Domain Generalization with Large Learning Rates Jian Zhang Lei Qi Yinghuan Shi Yang Gao 41 2 0 12 Sep 2023
Generalization error bounds for iterative learning algorithms with bounded updates Jingwen Fu Nanning Zheng 47 1 0 10 Sep 2023
Improving Resnet-9 Generalization Trained on Small Datasets Omar Mohamed Awad Habib Hajimolahoseini Michael Lim Gurpreet Gosal Walid Ahmed Yang Liu Gordon Deng 31 2 0 07 Sep 2023