On progressive sharpening, flat minima and generalisation

24 May 2023

Papers citing "On progressive sharpening, flat minima and generalisation"

21 / 21 papers shown

Title
On skip connections and normalisation layers in deep optimisation L. MacDonald Jack Valmadre Hemanth Saratchandran Simon Lucey ODL 56 2 0 10 Oct 2022
Understanding Edge-of-Stability Training Dynamics with a Minimalist Example Xingyu Zhu Zixuan Wang Xiang Wang Mo Zhou Rong Ge 98 37 0 07 Oct 2022
Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability Z. Li Zixuan Wang Jian Li 41 46 0 26 Jul 2022
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization Simone Bombari Mohammad Hossein Amani Marco Mondelli 53 26 0 20 May 2022
Understanding the unstable convergence of gradient descent Kwangjun Ahn J.N. Zhang S. Sra 72 60 0 03 Apr 2022
A Universal Law of Robustness via Isoperimetry Sébastien Bubeck Mark Sellke 38 218 0 26 May 2021
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 184 1,345 0 03 Oct 2020
Relative Flatness and Generalization Henning Petzka Michael Kamp Linara Adilova C. Sminchisescu Mario Boley 73 78 0 03 Jan 2020
Fantastic Generalization Measures and Where to Find Them Yiding Jiang Behnam Neyshabur H. Mobahi Dilip Krishnan Samy Bengio AI4CE 129 606 0 04 Dec 2019
Robust Learning with Jacobian Regularization Judy Hoffman Daniel A. Roberts Sho Yaida OOD AAML 51 167 0 07 Aug 2019
Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks Mahyar Fazlyab Alexander Robey Hamed Hassani M. Morari George J. Pappas 87 456 0 12 Jun 2019
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 214 1,270 0 04 Oct 2018
Regularisation of Neural Networks by Enforcing Lipschitz Continuity Henry Gouk E. Frank Bernhard Pfahringer M. Cree 170 477 0 12 Apr 2018
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 273 9,759 0 25 Oct 2017
Sharp asymptotic and finite-sample rates of convergence of empirical measures in Wasserstein distance Jonathan Niles-Weed Francis R. Bach 187 421 0 01 Jul 2017
Spectrally-normalized margin bounds for neural networks Peter L. Bartlett Dylan J. Foster Matus Telgarsky ODL 199 1,217 0 26 Jun 2017
Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Gintare Karolina Dziugaite Daniel M. Roy 106 813 0 31 Mar 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 334 4,625 0 10 Nov 2016
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys Pratik Chaudhari A. Choromańska Stefano Soatto Yann LeCun Carlo Baldassi C. Borgs J. Chayes Levent Sagun R. Zecchina ODL 94 773 0 06 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 419 2,936 0 15 Sep 2016
Identity Mappings in Deep Residual Networks Kaiming He Xinming Zhang Shaoqing Ren Jian Sun 350 10,180 0 16 Mar 2016