Averaging Weights Leads to Wider Optima and Better Generalization

14 March 2018

Dmitry Vetrov

Papers citing "Averaging Weights Leads to Wider Optima and Better Generalization"

50 / 366 papers shown

Title
BARTSmiles: Generative Masked Language Models for Molecular Representations Gayane Chilingaryan Hovhannes Tamoyan Ani Tevosyan N. Babayan L. Khondkaryan Karen Hambardzumyan Zaven Navoyan Hrant Khachatrian Armen Aghajanyan SSL 35 25 0 29 Nov 2022
Cross-Domain Ensemble Distillation for Domain Generalization Kyung-Jin Lee Sungyeon Kim Suha Kwak FedML OOD 26 38 0 25 Nov 2022
Indian Commercial Truck License Plate Detection and Recognition for Weighbridge Automation Siddharth Agrawal Keyur D. Joshi 35 4 0 23 Nov 2022
Improving Robust Generalization by Direct PAC-Bayesian Bound Minimization Zifa Wang Nan Ding Tomer Levinboim Xi Chen Radu Soricut AAML 35 5 0 22 Nov 2022
Pushing the Limits of Asynchronous Graph-based Object Detection with Event Cameras Daniel Gehrig Davide Scaramuzza GNN 24 29 0 22 Nov 2022
Non-reversible Parallel Tempering for Deep Posterior Approximation Wei Deng Qian Zhang Qi Feng F. Liang Guang Lin 26 4 0 20 Nov 2022
Mechanistic Mode Connectivity Ekdeep Singh Lubana Eric J. Bigelow Robert P. Dick David M. Krueger Hidenori Tanaka 32 45 0 15 Nov 2022
REPAIR: REnormalizing Permuted Activations for Interpolation Repair Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 46 94 0 15 Nov 2022
MEAL: Stable and Active Learning for Few-Shot Prompting Abdullatif Köksal Timo Schick Hinrich Schütze 24 25 0 15 Nov 2022
Learning to Annotate Part Segmentation with Gradient Matching Yu Yang Xiaotian Cheng Hakan Bilen Xiangyang Ji GAN 32 7 0 06 Nov 2022
Quantifying Model Uncertainty for Semantic Segmentation using Operators in the RKHS Rishabh Singh José C. Príncipe UQCV 33 3 0 03 Nov 2022
Circling Back to Recurrent Models of Language Gábor Melis 40 0 0 03 Nov 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Yaqing Wang Sahaj Agarwal Subhabrata Mukherjee Xiaodong Liu Jing Gao Ahmed Hassan Awadallah Jianfeng Gao MoE 22 118 0 31 Oct 2022
Symmetries, flat minima, and the conserved quantities of gradient flow Bo Zhao I. Ganev Robin Walters Rose Yu Nima Dehmamy 47 16 0 31 Oct 2022
Towards Generalized Few-Shot Open-Set Object Detection Binyi Su Hua Zhang Jingzhi Li Zhongjun Zhou 51 9 0 28 Oct 2022
Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition Steven Vander Eeckt Hugo Van hamme CLL MoMe 61 14 0 27 Oct 2022
Sufficient Invariant Learning for Distribution Shift Taero Kim Sungjun Lim Kyungwoo Song OOD 31 2 0 24 Oct 2022
On the optimization and pruning for Bayesian deep learning X. Ke Yanan Fan BDL UQCV 35 1 0 24 Oct 2022
Revisiting Checkpoint Averaging for Neural Machine Translation Yingbo Gao Christian Herold Zijian Yang Hermann Ney MoMe 27 11 0 21 Oct 2022
lo-fi: distributed fine-tuning without communication Mitchell Wortsman Suchin Gururangan Shen Li Ali Farhadi Ludwig Schmidt Michael G. Rabbat Ari S. Morcos 32 24 0 19 Oct 2022
Scaling Adversarial Training to Large Perturbation Bounds Sravanti Addepalli Samyak Jain Gaurang Sriramanan R. Venkatesh Babu AAML 33 22 0 18 Oct 2022
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models Nikolaos Dimitriadis P. Frossard Franccois Fleuret 29 25 0 18 Oct 2022
RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging A. Jaiswal Kumar Ashutosh Justin F. Rousseau Yifan Peng Zhangyang Wang Ying Ding 20 9 0 15 Oct 2022
Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networks A. K. Akash Sixu Li Nicolas García Trillos 34 12 0 13 Oct 2022
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling Haw-Shiuan Chang Ruei-Yao Sun Kathryn Ricci Andrew McCallum 43 14 0 10 Oct 2022
Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning Donald Shenaj Eros Fani Marco Toldo Debora Caldarola A. Tavera Umberto Michieli Marco Ciccone Pietro Zanuttigh Barbara Caputo FedML 29 39 0 05 Oct 2022
Stability Analysis and Generalization Bounds of Adversarial Training Jiancong Xiao Yanbo Fan Ruoyu Sun Jue Wang Zhimin Luo AAML 32 30 0 03 Oct 2022
Adaptive Smoothness-weighted Adversarial Training for Multiple Perturbations with Its Stability Analysis Jiancong Xiao Zeyu Qin Yanbo Fan Baoyuan Wu Jue Wang Zhimin Luo AAML 31 7 0 02 Oct 2022
Stop Wasting My Time! Saving Days of ImageNet and BERT Training with Latest Weight Averaging Jean Kaddour MoMe 3DH 24 39 0 29 Sep 2022
Learning Gradient-based Mixup towards Flatter Minima for Domain Generalization Danni Peng Sinno Jialin Pan 34 2 0 29 Sep 2022
Two-Tailed Averaging: Anytime, Adaptive, Once-in-a-While Optimal Weight Averaging for Better Generalization Gábor Melis MoMe 36 1 0 26 Sep 2022
Strong Transferable Adversarial Attacks via Ensembled Asymptotically Normal Distribution Learning Zhengwei Fang Rui Wang Tao Huang L. Jing AAML 32 5 0 24 Sep 2022
Random initialisations performing above chance and how to find them Frederik Benzing Simon Schug Robert Meier J. Oswald Yassir Akram Nicolas Zucchet Laurence Aitchison Angelika Steger ODL 35 24 0 15 Sep 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 316 0 11 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 31 4 0 06 Sep 2022
Investigating the Impact of Model Misspecification in Neural Simulation-based Inference Patrick W Cannon Daniel Ward Sebastian M. Schmon 25 34 0 05 Sep 2022
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey Yanbei Chen Massimiliano Mancini Xiatian Zhu Zeynep Akata 45 113 0 24 Aug 2022
A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspective Chanwoo Park Sangdoo Yun Sanghyuk Chun AAML 21 32 0 21 Aug 2022
Interpretable Uncertainty Quantification in AI for HEP Thomas Y. Chen B. Dey A. Ghosh Michael Kagan Brian D. Nord Nesar Ramachandra 33 7 0 05 Aug 2022
Learning Hyper Label Model for Programmatic Weak Supervision Renzhi Wu Sheng Chen Jieyu Zhang Xu Chu 26 16 0 27 Jul 2022
LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinity Martin Gubri Maxime Cordy Mike Papadakis Yves Le Traon Koushik Sen AAML 35 51 0 26 Jul 2022
Learning from Data with Noisy Labels Using Temporal Self-Ensemble Jun Ho Lee J. Baik Taebaek Hwang J. Choi NoLa 28 1 0 21 Jul 2022
Scene Text Recognition with Permuted Autoregressive Sequence Models Darwin Bautista Rowel Atienza 26 169 0 14 Jul 2022
YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors Chien-Yao Wang Alexey Bochkovskiy H. Liao ObjD 27 6,252 0 06 Jul 2022
Federated Self-supervised Learning for Video Understanding Yasar Abbas Ur Rehman Yan Gao Jiajun Shen Pedro Porto Buarque de Gusmão Nicholas D. Lane FedML 36 15 0 05 Jul 2022
Effective training-time stacking for ensembling of deep neural networks P. Proskura Alexey Zaytsev 17 6 0 27 Jun 2022
When Does Re-initialization Work? Sheheryar Zaidi Tudor Berariu Hyunjik Kim J. Bornschein Claudia Clopath Yee Whye Teh Razvan Pascanu 40 10 0 20 Jun 2022
Uncertainty-aware Evaluation of Time-Series Classification for Online Handwriting Recognition with Domain Shift Andreas Klass Sven M. Lorenz M. Lauer-Schmaltz David Rügamer Bernd Bischl Christopher Mutschler Felix Ott 34 10 0 17 Jun 2022
A Closer Look at Smoothness in Domain Adversarial Training Harsh Rangwani Sumukh K Aithal Mayank Mishra Arihant Jain R. Venkatesh Babu 35 119 0 16 Jun 2022
Federated Learning with Uncertainty via Distilled Predictive Distributions Shreyansh P. Bhatt Aishwarya Gupta Piyush Rai FedML 26 11 0 15 Jun 2022