v1v2v3 (latest)

Averaging Weights Leads to Wider Optima and Better Generalization

14 March 2018

Dmitry Vetrov

Papers citing "Averaging Weights Leads to Wider Optima and Better Generalization"

50 / 1,040 papers shown

Title
No One Representation to Rule Them All: Overlapping Features of Training Methods Raphael Gontijo-Lopes Yann N. Dauphin E. D. Cubuk 101 65 0 20 Oct 2021
Improving Robustness using Generated Data Sven Gowal Sylvestre-Alvise Rebuffi Olivia Wiles Florian Stimberg D. A. Calian Timothy A. Mann 122 302 0 18 Oct 2021
Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector Guoliang Lin Hanlu Chu Hanjiang Lai MoMe CLL 92 50 0 15 Oct 2021
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization Prafulla Kumar Choubey Alexander R. Fabbri Jesse Vig Chien-Sheng Wu Wenhao Liu Nazneen Rajani HILM 71 18 0 14 Oct 2021
Multi-ACCDOA: Localizing and Detecting Overlapping Sounds from the Same Class with Auxiliary Duplicating Permutation Invariant Training Kazuki Shimada Yuichiro Koyama Shusuke Takahashi Naoya Takahashi E. Tsunoo Yuki Mitsufuji 72 66 0 14 Oct 2021
Study of positional encoding approaches for Audio Spectrogram Transformers L. Pepino Pablo Riera Luciana Ferrer ViT 48 7 0 13 Oct 2021
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 102 238 0 12 Oct 2021
Momentum Centering and Asynchronous Update for Adaptive Gradient Methods Juntang Zhuang Yifan Ding Tommy M. Tang Nicha Dvornek S. Tatikonda James S. Duncan ODL 53 4 0 11 Oct 2021
Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks Hanxun Huang Yisen Wang S. Erfani Quanquan Gu James Bailey Xingjun Ma AAML TPM 139 102 0 07 Oct 2021
Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting Chengyu Dong Liyuan Liu Jingbo Shang NoLa AAML 119 20 0 07 Oct 2021
Improving Adversarial Robustness for Free with Snapshot Ensemble Yihao Wang AAML UQCV 36 1 0 07 Oct 2021
Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective Luca Scimeca Seong Joon Oh Sanghyuk Chun Michael Poli Sangdoo Yun OOD 590 54 0 06 Oct 2021
Geometric Transformers for Protein Interface Contact Prediction Alex Morehead Chen Chen Jianlin Cheng 136 29 0 06 Oct 2021
Batch size-invariance for policy optimization Jacob Hilton K. Cobbe John Schulman 120 14 0 01 Oct 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 282 500 0 01 Oct 2021
Q-Net: A Quantitative Susceptibility Mapping-based Deep Neural Network for Differential Diagnosis of Brain Iron Deposition in Hemochromatosis Soheil Zabihi E. Rahimian Soumya Sharma S. Sethi S. Gharabaghi A. Asif E. Haacke M. Jog Arash Mohammadi 20 1 0 01 Oct 2021
Perturbated Gradients Updating within Unit Space for Deep Learning Ching-Hsun Tseng Liu Cheng Shin-Jye Lee Xiaojun Zeng 111 5 0 01 Oct 2021
Training on Test Data with Bayesian Adaptation for Covariate Shift Aurick Zhou Sergey Levine OOD TTA 112 13 0 27 Sep 2021
Reduced-Lead ECG Classifier Model Trained with DivideMix and Model Ensemble Hiroshi Seki Takashi Nakano Koshiro Ikeda S. Hirooka Takaaki Kawasaki Mitsutomo Yamada Shumpei Saito T. Yamakawa Shimpei Ogawa 27 3 0 24 Sep 2021
A Physics inspired Functional Operator for Model Uncertainty Quantification in the RKHS Rishabh Singh José C. Príncipe 45 4 0 22 Sep 2021
A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentation Robin Camarasa D. Bos J. Hendrikse P. Nederkoorn D. Epidemiology D. Neurology Department of Computer Science UQCV 80 12 0 22 Sep 2021
iRNN: Integer-only Recurrent Neural Network Eyyub Sari Vanessa Courville V. Nia MQ 85 4 0 20 Sep 2021
Dynamic Neural Diversification: Path to Computationally Sustainable Neural Networks Alexander Kovalenko Pavel Kordík Magda Friedjungová 47 1 0 20 Sep 2021
Fine-Context Shadow Detection using Shadow Removal Jeya Maria Jose Valanarasu Vishal M. Patel 151 15 0 20 Sep 2021
Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression Masaki Morimoto Kai Fukami R. Maulik Ricardo Vinuesa K. Fukagata UQCV 84 31 0 16 Sep 2021
Connecting Low-Loss Subspace for Personalized Federated Learning S. Hahn Minwoo Jeong Junghye Lee FedML 94 19 0 16 Sep 2021
ARCH: Efficient Adversarial Regularized Training with Caching Simiao Zuo Chen Liang Haoming Jiang Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen T. Zhao AAML 78 3 0 15 Sep 2021
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques Shiyu Tang Ruihao Gong Yan Wang Aishan Liu Jiakai Wang ... Xianglong Liu Basel Alomair Alan Yuille Philip Torr Dacheng Tao VLM AAML 96 108 0 11 Sep 2021
Efficiently Identifying Task Groupings for Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 301 258 1 10 Sep 2021
Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-based Encoder Elman Mansimov Yi Zhang 205 15 0 09 Sep 2021
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization Alexandre Ramé Corentin Dancette Matthieu Cord OOD 146 210 0 07 Sep 2021
ISyNet: Convolutional Neural Networks design for AI accelerator Alexey Letunovskiy Vladimir Korviakov V. Polovnikov Anastasiia Kargapoltseva I. Mazurenko Yepan Xiong 104 1 0 04 Sep 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 219 741 0 04 Sep 2021
Bridged Adversarial Training Hoki Kim Woojin Lee Sungyoon Lee Jaewook Lee AAML GAN 65 9 0 25 Aug 2021
MS-DARTS: Mean-Shift Based Differentiable Architecture Search J. Hsieh Ming-Ching Chang Ping-Yang Chen Santanu Santra Cheng-Han Chou Chih-Sheng Huang OOD 41 2 0 23 Aug 2021
Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation Antyanta Bangunharcana Jae-Won Cho Seokju Lee In So Kweon Kyung-soo Kim Soohyun Kim 54 71 0 12 Aug 2021
Expressive Power and Loss Surfaces of Deep Learning Models S. Dube 36 0 0 08 Aug 2021
FPB: Feature Pyramid Branch for Person Re-Identification Suofei Zhang Zirui Yin Xiofu Wu Kun Wang Quan Zhou Bin Kang CVBM 50 12 0 04 Aug 2021
Real-Time Anchor-Free Single-Stage 3D Detection with IoU-Awareness Runzhou Ge Zhuangzhuang Ding Yihan Hu Wenxin Shao Li Huang Kun Li Qiang Liu 3DPC 93 8 0 29 Jul 2021
Taxonomizing local versus global structure in neural network loss landscapes Yaoqing Yang Liam Hodgkinson Ryan Theisen Joe Zou Joseph E. Gonzalez Kannan Ramchandran Michael W. Mahoney 118 37 0 23 Jul 2021
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion D. Kunin Javier Sagastuy-Breña Lauren Gillespie Eshed Margalit Hidenori Tanaka Surya Ganguli Daniel L. K. Yamins 93 20 0 19 Jul 2021
Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machines Byung-Hak Kim Varun Ganapathi 58 39 0 10 Jul 2021
L2M: Practical posterior Laplace approximation with optimization-driven second moment estimation C. Perone Roberto Silveira Thomas S. Paula ODL UQCV 59 2 0 09 Jul 2021
Federated Learning for Multi-Center Imaging Diagnostics: A Study in Cardiovascular Disease Akis Linardos Kaisar Kushibar S. Walsh P. Gkontra Karim Lekadir FedML 85 70 0 07 Jul 2021
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity A. Davtyan Sepehr Sameni L. Cerkezi Givi Meishvili Adam Bielski Paolo Favaro ODL 165 3 0 07 Jul 2021
Oriental Language Recognition (OLR) 2020: Summary and Analysis Jing Li Binling Wang Yiming Zhi Zheng Li Lin Li Q. Hong Dong Wang 54 11 0 05 Jul 2021
What can linear interpolation of neural network loss landscapes tell us? Tiffany J. Vlaar Jonathan Frankle MoMe 74 28 0 30 Jun 2021
Real-time Neural Radiance Caching for Path Tracing Thomas Müller Fabrice Rousselle Jan Novák A. Keller 3DH AI4CE 121 167 0 23 Jun 2021
Dangers of Bayesian Model Averaging under Covariate Shift Pavel Izmailov Patrick K. Nicholson Sanae Lotfi A. Wilson OOD UQCV BDL 151 46 0 22 Jun 2021
Rethinking Adam: A Twofold Exponential Moving Average Approach Yizhou Wang Yue Kang Can Qin Huan Wang Yi Xu Yulun Zhang Y. Fu ODL 70 7 0 22 Jun 2021