Averaging Weights Leads to Wider Optima and Better Generalization

14 March 2018

Dmitry Vetrov

Papers citing "Averaging Weights Leads to Wider Optima and Better Generalization"

50 / 366 papers shown

Title
Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning Daniel Grießhaber J. Maucher Ngoc Thang Vu 19 46 0 04 Dec 2020
A Random Matrix Theory Approach to Damping in Deep Learning Diego Granziol Nicholas P. Baskerville AI4CE ODL 29 2 0 15 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 77 670 0 06 Nov 2020
Measuring and Harnessing Transference in Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 28 15 0 29 Oct 2020
PEP: Parameter Ensembling by Perturbation Alireza Mehrtash Purang Abolmaesumi Polina Golland Tina Kapur Demian Wassermann W. Wells 25 10 0 24 Oct 2020
Combining Ensembles and Data Augmentation can Harm your Calibration Yeming Wen Ghassen Jerfel Rafael Muller Michael W. Dusenberry Jasper Snoek Balaji Lakshminarayanan Dustin Tran UQCV 32 63 0 19 Oct 2020
Consumer Behaviour in Retail: Next Logical Purchase using Deep Neural Network Ankur Verma 8 6 0 14 Oct 2020
Offer Personalization using Temporal Convolution Network and Optimization Ankur Verma 18 1 0 14 Oct 2020
Regularizing Neural Networks via Adversarial Model Perturbation Yaowei Zheng Richong Zhang Yongyi Mao AAML 30 95 0 10 Oct 2020
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples Sven Gowal Chongli Qin J. Uesato Timothy A. Mann Pushmeet Kohli AAML 17 324 0 07 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 110 1,278 0 03 Oct 2020
End-to-End Training of CNN Ensembles for Person Re-Identification Ayse Serbetci Y. S. Akgul 16 23 0 03 Oct 2020
SESQA: semi-supervised learning for speech quality assessment Joan Serrà Jordi Pons Santiago Pascual 18 42 0 01 Oct 2020
Conversational Semantic Parsing Armen Aghajanyan Jean Maillard Akshat Shrivastava K. Diedrick Mike Haeger ... Yashar Mehdad Ves Stoyanov Anuj Kumar M. Lewis S. Gupta 19 48 0 28 Sep 2020
Adversarial Training with Stochastic Weight Average Joong-won Hwang Youngwan Lee Sungchan Oh Yuseok Bae OOD AAML 29 11 0 21 Sep 2020
Kaggle forecasting competitions: An overlooked learning opportunity Casper Solheim Bojer Jens Peder Meldgaard AI4TS 16 207 0 16 Sep 2020
Ramifications of Approximate Posterior Inference for Bayesian Deep Learning in Adversarial and Out-of-Distribution Settings John Mitros A. Pakrashi Brian Mac Namee UQCV 26 2 0 03 Sep 2020
VarifocalNet: An IoU-aware Dense Object Detector Haoyang Zhang Ying Wang Feras Dayoub Niko Sünderhauf ObjD 21 665 0 31 Aug 2020
Obtaining Adjustable Regularization for Free via Iterate Averaging Jingfeng Wu Vladimir Braverman Lin F. Yang 30 2 0 15 Aug 2020
A community-powered search of machine learning strategy space to find NMR property prediction models Lars A. Bratholm W. Gerrard Brandon M. Anderson Shaojie Bai Sunghwan Choi ... A. Torrubia Devin Willmott C. Butts David R. Glowacki Kaggle participants 16 16 0 13 Aug 2020
DS-Sync: Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training Weiyan Wang Cengguang Zhang Liu Yang Kai Chen Kun Tan 29 12 0 07 Jul 2020
Directional Pruning of Deep Neural Networks Shih-Kang Chao Zhanyu Wang Yue Xing Guang Cheng ODL 21 33 0 16 Jun 2020
Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training Diego Granziol S. Zohren Stephen J. Roberts ODL 37 49 0 16 Jun 2020
Calibrating Deep Neural Network Classifiers on Out-of-Distribution Datasets Zhihui Shao Jianyi Yang Shaolei Ren OODD 35 11 0 16 Jun 2020
Hindsight Logging for Model Training Rolando Garcia Eric Liu Vikram Sreekanti Bobby Yan Anusha Dandamudi Joseph E. Gonzalez J. M. Hellerstein Koushik Sen VLM 27 10 0 12 Jun 2020
A benchmark study on reliable molecular supervised learning via Bayesian learning Doyeong Hwang Grace Lee Hanseok Jo Seyoul Yoon Seongok Ryu 22 9 0 12 Jun 2020
An Overview of Deep Semi-Supervised Learning Yassine Ouali C´eline Hudelot Myriam Tami SSL HAI 27 294 0 09 Jun 2020
Detached Error Feedback for Distributed SGD with Random Sparsification An Xu Heng-Chiao Huang 39 9 0 11 Apr 2020
Orthogonal Over-Parameterized Training Weiyang Liu Rongmei Lin Zhen Liu James M. Rehg Liam Paull Li Xiong Le Song Adrian Weller 32 41 0 09 Apr 2020
Iterative Averaging in the Quest for Best Test Error Diego Granziol Xingchen Wan Samuel Albanie Stephen J. Roberts 10 3 0 02 Mar 2020
Towards Robust and Reproducible Active Learning Using Neural Networks Prateek Munjal Nasir Hayat Munawar Hayat J. Sourati Shadab Khan UQCV 25 67 0 21 Feb 2020
Bayesian Deep Learning and a Probabilistic Perspective of Generalization A. Wilson Pavel Izmailov UQCV BDL OOD 24 639 0 20 Feb 2020
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning Arsenii Ashukha Alexander Lyzhov Dmitry Molchanov Dmitry Vetrov UQCV FedML 33 314 0 15 Feb 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 8 18 0 12 Feb 2020
No Routing Needed Between Capsules Adam Byerly T. Kalganova I. Dear 42 67 0 24 Jan 2020
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning Paola Cascante-Bonilla Fuwen Tan Yanjun Qi Vicente Ordonez ODL 47 23 0 16 Jan 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 38 55 0 07 Jan 2020
Big Transfer (BiT): General Visual Representation Learning Alexander Kolesnikov Lucas Beyer Xiaohua Zhai J. Puigcerver Jessica Yung Sylvain Gelly N. Houlsby MQ 100 1,183 0 24 Dec 2019
TRADI: Tracking deep neural network weight distributions for uncertainty estimation Gianni Franchi Andrei Bursuc Emanuel Aldea Séverine Dubuisson Isabelle Bloch UQCV 26 51 0 24 Dec 2019
Deep Ensembles: A Loss Landscape Perspective Stanislav Fort Huiyi Hu Balaji Lakshminarayanan OOD UQCV 29 617 0 05 Dec 2019
Information-Theoretic Local Minima Characterization and Regularization Zhiwei Jia Hao Su 27 19 0 19 Nov 2019
Disentangle, align and fuse for multimodal and semi-supervised image segmentation A. Chartsias G. Papanastasiou Chengjia Wang S. Semple D. Newby R. Dharmakumar Sotirios A. Tsaftaris 24 13 0 11 Nov 2019
Segmenting Ships in Satellite Imagery With Squeeze and Excitation U-Net R. Venkatesh Anand Metha SSeg 21 4 0 27 Oct 2019
Self-Correction for Human Parsing Peike Li Yunqiu Xu Yunchao Wei Yezhou Yang 27 325 0 22 Oct 2019
BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization Maximilian Balandat Brian Karrer Daniel R. Jiang Sam Daulton Benjamin Letham A. Wilson E. Bakshy 32 93 0 14 Oct 2019
Towards Understanding the Transferability of Deep Representations Hong Liu Mingsheng Long Jianmin Wang Michael I. Jordan 30 25 0 26 Sep 2019
On Model Stability as a Function of Random Seed Pranava Madhyastha Dhruv Batra 42 61 0 23 Sep 2019
Visualizing and Understanding the Effectiveness of BERT Y. Hao Li Dong Furu Wei Ke Xu 27 181 0 15 Aug 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 48 719 0 19 Jul 2019
Subspace Inference for Bayesian Deep Learning Pavel Izmailov Wesley J. Maddox Polina Kirichenko T. Garipov Dmitry Vetrov A. Wilson UQCV BDL 38 142 0 17 Jul 2019