Averaging Weights Leads to Wider Optima and Better Generalization

14 March 2018

Dmitry Vetrov

Papers citing "Averaging Weights Leads to Wider Optima and Better Generalization"

50 / 366 papers shown

Title
Study of positional encoding approaches for Audio Spectrogram Transformers L. Pepino Pablo Riera Luciana Ferrer ViT 28 6 0 13 Oct 2021
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 39 217 0 12 Oct 2021
Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks Hanxun Huang Yisen Wang S. Erfani Quanquan Gu James Bailey Xingjun Ma AAML TPM 46 100 0 07 Oct 2021
Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting Chengyu Dong Liyuan Liu Jingbo Shang NoLa AAML 56 18 0 07 Oct 2021
Improving Adversarial Robustness for Free with Snapshot Ensemble Yihao Wang AAML UQCV 17 1 0 07 Oct 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 212 487 0 01 Oct 2021
Perturbated Gradients Updating within Unit Space for Deep Learning Ching-Hsun Tseng Liu Cheng Shin-Jye Lee Xiaojun Zeng 45 5 0 01 Oct 2021
A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentation Robin Camarasa D. Bos J. Hendrikse P. Nederkoorn D. Epidemiology D. Neurology Department of Computer Science UQCV 24 12 0 22 Sep 2021
iRNN: Integer-only Recurrent Neural Network Eyyub Sari Vanessa Courville V. Nia MQ 56 4 0 20 Sep 2021
Connecting Low-Loss Subspace for Personalized Federated Learning S. Hahn Minwoo Jeong Junghye Lee FedML 24 18 0 16 Sep 2021
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization Alexandre Ramé Corentin Dancette Matthieu Cord OOD 40 204 0 07 Sep 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 64 691 0 04 Sep 2021
Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation Antyanta Bangunharcana Jae-Won Cho Seokju Lee In So Kweon Kyung-soo Kim Soohyun Kim 16 67 0 12 Aug 2021
FPB: Feature Pyramid Branch for Person Re-Identification Suofei Zhang Zirui Yin Xiofu Wu Kun Wang Quan Zhou Bin Kang CVBM 19 12 0 04 Aug 2021
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion D. Kunin Javier Sagastuy-Breña Lauren Gillespie Eshed Margalit Hidenori Tanaka Surya Ganguli Daniel L. K. Yamins 31 15 0 19 Jul 2021
Federated Learning for Multi-Center Imaging Diagnostics: A Study in Cardiovascular Disease Akis Linardos Kaisar Kushibar S. Walsh P. Gkontra Karim Lekadir FedML 25 62 0 07 Jul 2021
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity A. Davtyan Sepehr Sameni L. Cerkezi Givi Meishvili Adam Bielski Paolo Favaro ODL 53 2 0 07 Jul 2021
Oriental Language Recognition (OLR) 2020: Summary and Analysis Jing Li Binling Wang Yiming Zhi Zheng Li Lin Li Q. Hong Dong Wang 24 10 0 05 Jul 2021
What can linear interpolation of neural network loss landscapes tell us? Tiffany J. Vlaar Jonathan Frankle MoMe 30 27 0 30 Jun 2021
Real-time Neural Radiance Caching for Path Tracing Thomas Müller Fabrice Rousselle Jan Novák A. Keller 3DH AI4CE 25 155 0 23 Jun 2021
Humble Teachers Teach Better Students for Semi-Supervised Object Detection Yihe Tang Weifeng Chen Yijun Luo Yuting Zhang 36 177 0 19 Jun 2021
Effective Evaluation of Deep Active Learning on Image Classification Tasks Nathan Beck D. Sivasubramanian Apurva Dani Ganesh Ramakrishnan Rishabh K. Iyer VLM 20 38 0 16 Jun 2021
ScheduleNet: Learn to solve multi-agent scheduling problems with reinforcement learning Junyoung Park Sanjar Bakhtiyar Jinkyoo Park 18 38 0 06 Jun 2021
Efficient and Accurate Gradients for Neural SDEs Patrick Kidger James Foster Xuechen Li Terry Lyons DiffM 24 60 0 27 May 2021
Denoising Noisy Neural Networks: A Bayesian Approach with Compensation Yulin Shao Soung Chang Liew Deniz Gunduz 56 14 0 22 May 2021
Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report Andrey D. Ignatov Grigory Malivenko Radu Timofte Sheng Chen Xin Xia ... K. Lyda L. Khojoyan Abhishek Thanki Sayak Paul Shahid Siddiqui MQ 21 20 0 17 May 2021
Self-supervised Augmentation Consistency for Adapting Semantic Segmentation Nikita Araslanov Stefan Roth 44 227 0 30 Apr 2021
Post-training deep neural network pruning via layer-wise calibration Ivan Lazarevich Alexander Kozlov Nikita Malinin 3DPC 18 25 0 30 Apr 2021
SelfReg: Self-supervised Contrastive Regularization for Domain Generalization Daehee Kim Seunghyun Park Jinkyu Kim Jaekoo Lee OOD SSL 65 264 0 20 Apr 2021
Relating Adversarially Robust Generalization to Flat Minima David Stutz Matthias Hein Bernt Schiele OOD 32 65 0 09 Apr 2021
AST: Audio Spectrogram Transformer Yuan Gong Yu-An Chung James R. Glass ViT 31 830 0 05 Apr 2021
Domain Generalization: A Survey Kaiyang Zhou Ziwei Liu Yu Qiao Tao Xiang Chen Change Loy OOD AI4CE 75 980 0 03 Mar 2021
Fixing Data Augmentation to Improve Adversarial Robustness Sylvestre-Alvise Rebuffi Sven Gowal D. A. Calian Florian Stimberg Olivia Wiles Timothy A. Mann AAML 36 269 0 02 Mar 2021
A Multiclass Boosting Framework for Achieving Fast and Provable Adversarial Robustness Jacob D. Abernethy Pranjal Awasthi Satyen Kale AAML 27 6 0 01 Mar 2021
Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling Gregory W. Benton Wesley J. Maddox Sanae Lotfi A. Wilson UQCV 25 67 0 25 Feb 2021
Provable Super-Convergence with a Large Cyclical Learning Rate Samet Oymak 33 12 0 22 Feb 2021
Learning Neural Network Subspaces Mitchell Wortsman Maxwell Horton Carlos Guestrin Ali Farhadi Mohammad Rastegari UQCV 27 85 0 20 Feb 2021
DEUP: Direct Epistemic Uncertainty Prediction Salem Lahlou Moksh Jain Hadi Nekoei V. Butoi Paul Bertin Jarrid Rector-Brooks Maksym Korablyov Yoshua Bengio PER UQLM UQCV UD 204 81 0 16 Feb 2021
Adversarially Robust Kernel Smoothing Jia-Jie Zhu Christina Kouridi Yassine Nemmour Bernhard Schölkopf 28 7 0 16 Feb 2021
Low Curvature Activations Reduce Overfitting in Adversarial Training Vasu Singla Sahil Singla David Jacobs S. Feizi AAML 32 45 0 15 Feb 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 19 76 0 09 Feb 2021
On the Reproducibility of Neural Network Predictions Srinadh Bhojanapalli Kimberly Wilber Andreas Veit A. S. Rawat Seungyeon Kim A. Menon Sanjiv Kumar 29 35 0 05 Feb 2021
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation Yuan Gong Yu-An Chung James R. Glass VLM 104 144 0 02 Feb 2021
Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss Xue Yang Junchi Yan Qi Ming Wentao Wang Xiaopeng Zhang Qi Tian 113 399 0 28 Jan 2021
Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning Zhaowei Cai Avinash Ravichandran Subhransu Maji Charless C. Fowlkes Z. Tu Stefano Soatto 36 118 0 21 Jan 2021
A Survey on Ensemble Learning under the Era of Deep Learning Yongquan Yang Haijun Lv Ning Chen OOD 67 182 0 21 Jan 2021
LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification Ting Jiang Deqing Wang Leilei Sun Huayi Yang Zhengyang Zhao Fuzhen Zhuang VLM 128 136 0 09 Jan 2021
Combating Mode Collapse in GAN training: An Empirical Analysis using Hessian Eigenvalues Ricard Durall Avraam Chatzimichailidis P. Labus J. Keuper GAN 30 57 0 17 Dec 2020
FedADC: Accelerated Federated Learning with Drift Control Emre Ozfatura Kerem Ozfatura Deniz Gunduz FedML 43 37 0 16 Dec 2020
DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation R. A. Kamraoui Vinh-Thong Ta T. Tourdias Boris Mansencal J. V. Manjón Pierrick Coupé OOD 31 50 0 14 Dec 2020