Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance

17 July 2024

Yujun Yan

Huanrui Yang

Yaoqing Yang

ArXiv (abs)PDF HTML

Papers citing "Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance"

21 / 21 papers shown

Title
A Modern Look at the Relationship between Sharpness and Generalization Maksym Andriushchenko Francesco Croce Maximilian Müller Matthias Hein Nicolas Flammarion 3DH 111 63 0 14 Feb 2023
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 240 138 0 19 May 2022
Surrogate Gap Minimization Improves Sharpness-Aware Training Juntang Zhuang Boqing Gong Liangzhe Yuan Huayu Chen Hartwig Adam Nicha Dvornek S. Tatikonda James Duncan Ting Liu 91 157 0 15 Mar 2022
Diversity and Generalization in Neural Network Ensembles Luis A. Ortega Rafael Cabañas A. Masegosa FedML UQCV 71 45 0 26 Oct 2021
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity Shiwei Liu Tianlong Chen Zahra Atashgahi Xiaohan Chen Ghada Sokar Elena Mocanu Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu OOD 87 52 0 28 Jun 2021
Assessing Generalization of SGD via Disagreement Yiding Jiang Vaishnavh Nagarajan Christina Baek J. Zico Kolter 95 115 0 25 Jun 2021
ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks Jungmin Kwon Jeongseop Kim Hyunseong Park I. Choi 111 290 0 23 Feb 2021
SWAD: Domain Generalization by Seeking Flat Minima Junbum Cha Sanghyuk Chun Kyungjae Lee Han-Cheol Cho Seunghyun Park Yunsung Lee Sungrae Park MoMe 301 459 0 17 Feb 2021
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 199 1,359 0 03 Oct 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 115 287 0 01 Jun 2020
PyHessian: Neural Networks Through the Lens of the Hessian Z. Yao A. Gholami Kurt Keutzer Michael W. Mahoney ODL 74 304 0 16 Dec 2019
Deep Ensembles: A Loss Landscape Perspective Stanislav Fort Huiyi Hu Balaji Lakshminarayanan OOD UQCV 125 629 0 05 Dec 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 373 6,469 0 26 Sep 2019
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift Yaniv Ovadia Emily Fertig Jie Jessie Ren Zachary Nado D. Sculley Sebastian Nowozin Joshua V. Dillon Balaji Lakshminarayanan Jasper Snoek UQCV 187 1,705 0 06 Jun 2019
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations Dan Hendrycks Thomas G. Dietterich OOD VLM 191 3,455 0 28 Mar 2019
Improving Adversarial Robustness via Promoting Ensemble Diversity Tianyu Pang Kun Xu Chao Du Ning Chen Jun Zhu AAML 93 439 0 25 Jan 2019
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries Z. Yao A. Gholami Qi Lei Kurt Keutzer Michael W. Mahoney 69 167 0 22 Feb 2018
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks Behnam Neyshabur Srinadh Bhojanapalli Nathan Srebro 88 610 0 29 Jul 2017
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 842 5,847 0 05 Dec 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 433 2,946 0 15 Sep 2016
XGBoost: A Scalable Tree Boosting System Tianqi Chen Carlos Guestrin 817 39,062 0 09 Mar 2016