v1v2v3 (latest)

Averaging Weights Leads to Wider Optima and Better Generalization

14 March 2018

Dmitry Vetrov

Papers citing "Averaging Weights Leads to Wider Optima and Better Generalization"

50 / 1,040 papers shown

Title
Effective Gradient Sample Size via Variation Estimation for Accelerating Sharpness aware Minimization Jiaxin Deng Junbiao Pang Baochang Zhang Tian Wang 70 1 0 24 Feb 2024
Helen: Optimizing CTR Prediction Models with Frequency-wise Hessian Eigenvalue Regularization Zirui Zhu Yong Liu Zangwei Zheng Huifeng Guo Yang You 45 0 0 23 Feb 2024
On the Duality Between Sharpness-Aware Minimization and Adversarial Training Yihao Zhang Hangzhou He Jingyu Zhu Huanran Chen Yifei Wang Zeming Wei AAML 125 15 0 23 Feb 2024
Beyond Simple Averaging: Improving NLP Ensemble Performance with Topological-Data-Analysis-Based Weighting P. Proskura Alexey Zaytsev 90 0 0 22 Feb 2024
Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic Rishabh Bhardwaj Do Duc Anh Soujanya Poria MoMe 110 48 0 19 Feb 2024
The Effectiveness of Random Forgetting for Robust Generalization V. Ramkumar Bahram Zonooz Elahe Arani AAML 68 1 0 18 Feb 2024
Switch EMA: A Free Lunch for Better Flatness and Sharpness Siyuan Li Zicheng Liu Juanxi Tian Ge Wang Zedong Wang ... Cheng Tan Tao Lin Yang Liu Baigui Sun Stan Z. Li 66 6 0 14 Feb 2024
HYPO: Hyperspherical Out-of-Distribution Generalization Yifei Ming Haoyue Bai Julian Katz-Samuels Yixuan Li OODD OOD 81 10 0 12 Feb 2024
Curvature-Informed SGD via General Purpose Lie-Group Preconditioners Omead Brandon Pooladzandi Xi-Lin Li 88 8 0 07 Feb 2024
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm Zhanpeng Zhou Zijun Chen Yilan Chen Bo Zhang Junchi Yan MoMe 102 11 0 06 Feb 2024
How Good is a Single Basin? Kai Lion Lorenzo Noci Thomas Hofmann Gregor Bachmann UQCV 51 3 0 05 Feb 2024
Diffusive Gibbs Sampling Jiajun He Mingtian Zhang Brooks Paige José Miguel Hernández-Lobato David Barber 174 11 0 05 Feb 2024
Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate Can Jin Tong Che Hongwu Peng Yiyuan Li Dimitris N. Metaxas Marco Pavone 135 47 0 05 Feb 2024
eXplainable Bayesian Multi-Perspective Generative Retrieval EuiYul Song Philhoon Oh Sangryul Kim James Thorne BDL 54 0 0 04 Feb 2024
DFML: Decentralized Federated Mutual Learning Yasser H. Khalil A. Estiri Mahdi Beitollahi Nader Asadi S. Hemati Xu Li Guojun Zhang Xi Chen 27 1 0 02 Feb 2024
Training-time Neuron Alignment through Permutation Subspace for Improving Linear Mode Connectivity and Model Fusion Zexi Li Zhiqi Li Jie Lin Tao Shen Tao Lin Chao Wu 128 5 0 02 Feb 2024
Credal Learning Theory Michele Caprio Maryam Sultana Eleni Elia Fabio Cuzzolin FedML 116 15 0 01 Feb 2024
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts Anke Tang Li Shen Yong Luo Nan Yin Lefei Zhang Dacheng Tao MoMe 81 54 0 01 Feb 2024
RADIN: Souping on a Budget Thibaut Menes Olivier Risser-Maroix MoMe 104 1 0 31 Jan 2024
Explaining Predictive Uncertainty by Exposing Second-Order Effects Florian Bley Sebastian Lapuschkin Wojciech Samek G. Montavon 80 3 0 30 Jan 2024
Understanding Domain Generalization: A Noise Robustness Perspective Rui Qiao K. H. Low OOD 86 6 0 26 Jan 2024
Learning under Label Noise through Few-Shot Human-in-the-Loop Refinement Aaqib Saeed Dimitris Spathis Jungwoo Oh Edward Choi Ali Etemad NoLa 68 3 0 25 Jan 2024
DAFA: Distance-Aware Fair Adversarial Training Hyungyu Lee Saehyung Lee Hyemi Jang Junsung Park Ho Bae Sungroh Yoon 63 7 0 23 Jan 2024
A Precise Characterization of SGD Stability Using Loss Surface Geometry Gregory Dexter Borja Ocejo S. Keerthi Aman Gupta Ayan Acharya Rajiv Khanna MLT 78 0 0 22 Jan 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 179 104 0 22 Jan 2024
Adapters Mixup: Mixing Parameter-Efficient Adapters to Enhance the Adversarial Robustness of Fine-tuned Pre-trained Text Classifiers Tuc Nguyen Thai Le AAML SILM MoE 102 2 0 18 Jan 2024
Bag of Tricks to Boost Adversarial Transferability Zeliang Zhang Rongyi Zhu Wei Yao Xiaosen Wang Chenliang Xu AAML 86 11 0 16 Jan 2024
Robust Calibration For Improved Weather Prediction Under Distributional Shift Sankalp Gilda Neel Bhandari Wendy Mak Andrea Panizza UQCV OOD 26 1 0 08 Jan 2024
TSPP: A Unified Benchmarking Tool for Time-series Forecasting Jan Bkaczek Dmytro Zhylko Gilberto Titericz Sajad Darabi Jean-François Puget Izzy Putterman Dawid Majchrowski Anmol Gupta Kyle Kranen Pawel Morkisz AI4TS 92 1 0 28 Dec 2023
Enhancing Neural Training via a Correlated Dynamics Model Jonathan Brokman Roy Betser Rotem Turjeman Tom Berkov I. Cohen Guy Gilboa 54 3 0 20 Dec 2023
Doubly Perturbed Task Free Continual Learning Byung Hyun Lee Min-hwan Oh Se Young Chun 75 3 0 20 Dec 2023
Scaling Up Bayesian Neural Networks with Neural Networks Zahra Moslemi Yang Meng Shiwei Lan Babak Shahbaba BDL 73 1 0 19 Dec 2023
IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognition Xiaomeng Yang Zhi Qiao Yu Zhou DiffM 181 1 0 19 Dec 2023
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision Collin Burns Pavel Izmailov Jan Hendrik Kirchner Bowen Baker Leo Gao ... Adrien Ecoffet Manas Joglekar Jan Leike Ilya Sutskever Jeff Wu ELM 131 297 0 14 Dec 2023
Weighted Ensemble Models Are Strong Continual Learners Imad Eddine Marouf Subhankar Roy Enzo Tartaglione Stéphane Lathuilière CLL 109 22 0 14 Dec 2023
Mini-batch Gradient Descent with Buffer Haobo Qi Du Huang Yingqiu Zhu Danyang Huang Hansheng Wang 45 1 0 14 Dec 2023
Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Masks Mohammad-Javad Davari Eugene Belilovsky MoMe 99 71 0 11 Dec 2023
The Limits of Fair Medical Imaging AI In The Wild Yuzhe Yang Haoran Zhang Judy W. Gichoya Dina Katabi Marzyeh Ghassemi 77 6 0 11 Dec 2023
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion Anke Tang Li Shen Yong Luo Liang Ding Han Hu Bo Du Dacheng Tao MoMe 91 22 0 11 Dec 2023
Uncertainty-aware Surrogate Models for Airfoil Flow Simulations with Denoising Diffusion Probabilistic Models Qiang Liu Nils Thuerey DiffM AI4CE 84 18 0 08 Dec 2023
Open Domain Generalization with a Single Network by Regularization Exploiting Pre-trained Features Inseop Chung Kiyoon Yoo Nojun Kwak VLM 97 0 0 08 Dec 2023
Analyzing and Improving the Training Dynamics of Diffusion Models Tero Karras M. Aittala J. Lehtinen Janne Hellsten Timo Aila S. Laine 153 203 0 05 Dec 2023
Uncertainty in Graph Contrastive Learning with Bayesian Neural Networks Alexander M¨ollers Alexander Immer Elvin Isufi Vincent Fortuin SSL BDL UQCV 117 1 0 30 Nov 2023
Seg2Reg: Differentiable 2D Segmentation to 1D Regression Rendering for 360 Room Layout Reconstruction Cheng Sun Wei-En Tai Yu-Lin Shih Kuan-Wei Chen Yong-Jing Syu Kent Selwyn The Yu-Chiang Frank Wang Hwann-Tzong Chen 3DV 71 3 0 30 Nov 2023
Efficient Stitchable Task Adaptation Haoyu He Zizheng Pan Jing Liu Jianfei Cai Bohan Zhuang 120 3 0 29 Nov 2023
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 160 2 0 29 Nov 2023
Deployment of a Robust and Explainable Mortality Prediction Model: The COVID-19 Pandemic and Beyond Jacob R. Epifano Stephen Glass Ravichandran Ramachandran Sharad Patel A. Masino Ghulam Rasool 45 0 0 28 Nov 2023
Should We Learn Most Likely Functions or Parameters? Shikai Qiu Tim G. J. Rudner Sanyam Kapoor Andrew Gordon Wilson 46 6 0 27 Nov 2023
Parameter Exchange for Robust Dynamic Domain Generalization Luojun Lin Zhifeng Shen Zhishu Sun Yuanlong Yu Lei Zhang Weijie Chen OOD 117 6 0 23 Nov 2023
Large Learning Rates Improve Generalization: But How Large Are We Talking About? E. Lobacheva Eduard Pockonechnyy M. Kodryan Dmitry Vetrov AI4CE 33 0 0 19 Nov 2023