Asymptotic Unbiased Sample Sampling to Speed Up Sharpness-Aware Minimization

12 June 2024

Papers citing "Asymptotic Unbiased Sample Sampling to Speed Up Sharpness-Aware Minimization"

35 / 35 papers shown

Title
Stabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrix Junbiao Pang Tianyang Cai 97 1 0 14 Mar 2025
Effective Gradient Sample Size via Variation Estimation for Accelerating Sharpness aware Minimization Jiaxin Deng Junbiao Pang Baochang Zhang Tian Wang 65 1 0 24 Feb 2024
Momentum-SAM: Sharpness Aware Minimization without Computational Overhead Marlon Becker Frederick Altrock Benjamin Risse 108 6 0 22 Jan 2024
Flatness-Aware Minimization for Domain Generalization Xingxuan Zhang Renzhe Xu Han Yu Yancheng Dong Pengfei Tian Peng Cu 56 22 0 20 Jul 2023
An Adaptive Policy to Employ Sharpness-Aware Minimization Weisen Jiang Hansi Yang Yu Zhang James T. Kwok AAML 97 34 0 28 Apr 2023
InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning Ziheng Qin Kaidi Wang Zangwei Zheng Jianyang Gu Xiang Peng ... Daquan Zhou Lei Shang Baigui Sun Xuansong Xie Yang You 150 51 0 08 Mar 2023
K-SAM: Sharpness-Aware Minimization at the Speed of SGD Renkun Ni Ping Yeh-Chiang Jonas Geiping Micah Goldblum A. Wilson Tom Goldstein 32 9 0 23 Oct 2022
Towards Understanding Sharpness-Aware Minimization Maksym Andriushchenko Nicolas Flammarion AAML 82 141 0 13 Jun 2022
Sharpness-Aware Training for Free Jiawei Du Daquan Zhou Jiashi Feng Vincent Y. F. Tan Qiufeng Wang AAML 64 95 0 27 May 2022
Overcoming Oscillations in Quantization-Aware Training Markus Nagel Marios Fournarakis Yelysei Bondarenko Tijmen Blankevoort MQ 130 105 0 21 Mar 2022
Surrogate Gap Minimization Improves Sharpness-Aware Training Juntang Zhuang Boqing Gong Liangzhe Yuan Huayu Chen Hartwig Adam Nicha Dvornek S. Tatikonda James Duncan Ting Liu 59 154 0 15 Mar 2022
QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization Xiuying Wei Ruihao Gong Yuhang Li Xianglong Liu F. Yu MQ VLM 63 174 0 11 Mar 2022
Accelerating Deep Learning with Dynamic Data Pruning Ravi Raju Kyle Daruwalla Mikko H. Lipasti 18 31 0 24 Nov 2021
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 147 102 0 16 Oct 2021
Efficient Sharpness-aware Minimization for Improved Training of Neural Networks Jiawei Du Hanshu Yan Jiashi Feng Qiufeng Wang Liangli Zhen Rick Siow Mong Goh Vincent Y. F. Tan AAML 128 135 0 07 Oct 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training Mansheej Paul Surya Ganguli Gintare Karolina Dziugaite 103 456 0 15 Jul 2021
SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation Yanjie Li Sen Yang Peidong Liu Shoukui Zhang Yunxiao Wang Zhicheng Wang Wankou Yang Shutao Xia 3DH 60 126 0 07 Jul 2021
ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks Jungmin Kwon Jeongseop Kim Hyunseong Park I. Choi 79 289 0 23 Feb 2021
SWAD: Domain Generalization by Seeking Flat Minima Junbum Cha Sanghyuk Chun Kyungjae Lee Han-Cheol Cho Seunghyun Park Yunsung Lee Sungrae Park MoMe 273 449 0 17 Feb 2021
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 182 1,342 0 03 Oct 2020
Fantastic Generalization Measures and Where to Find Them Yiding Jiang Behnam Neyshabur H. Mobahi Dilip Krishnan Samy Bengio AI4CE 103 605 0 04 Dec 2019
Learned Step Size Quantization S. K. Esser J. McKinstry Deepika Bablani R. Appuswamy D. Modha MQ 69 798 0 21 Feb 2019
An Empirical Study of Example Forgetting during Deep Neural Network Learning Mariya Toneva Alessandro Sordoni Rémi Tachet des Combes Adam Trischler Yoshua Bengio Geoffrey J. Gordon 107 731 0 12 Dec 2018
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 107 1,654 0 14 Mar 2018
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Benoit Jacob S. Kligys Bo Chen Menglong Zhu Matthew Tang Andrew G. Howard Hartwig Adam Dmitry Kalenichenko MQ 136 3,111 0 15 Dec 2017
Improved Regularization of Convolutional Neural Networks with Cutout Terrance Devries Graham W. Taylor 101 3,756 0 15 Aug 2017
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets P. Chrabaszcz I. Loshchilov Frank Hutter SSeg OOD 125 644 0 27 Jul 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 139 1,250 0 27 Jun 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.1K 20,781 0 17 Apr 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 304 4,623 0 10 Nov 2016
Deep Pyramidal Residual Networks Dongyoon Han Jiwhan Kim Junmo Kim 93 692 0 10 Oct 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 390 2,934 0 15 Sep 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 274 8,082 0 13 Aug 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 299 7,967 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.7K 193,390 0 10 Dec 2015