v1v2 (latest)

Sharp Minima Can Generalize For Deep Nets

15 March 2017

Papers citing "Sharp Minima Can Generalize For Deep Nets"

50 / 61 papers shown

Title
An Analytical Characterization of Sloppiness in Neural Networks: Insights from Linear Models Jialin Mao Itay Griniasty Yan Sun Mark K. Transtrum James P. Sethna Pratik Chaudhari 99 0 0 13 May 2025
How Effective Can Dropout Be in Multiple Instance Learning ? Wenhui Zhu Peijie Qiu Xiwen Chen Zhangsihao Yang Aristeidis Sotiras Abolfazl Razi Yanjie Wang 71 0 0 21 Apr 2025
High-entropy Advantage in Neural Networks' Generalizability Entao Yang Wei Wei Yue Shang Ge Zhang AI4CE 102 0 0 17 Mar 2025
Meta Curvature-Aware Minimization for Domain Generalization Zhaoyu Chen Yiwen Ye Feilong Tang Yongsheng Pan Yong-quan Xia BDL 399 1 0 16 Dec 2024
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks Jim Zhao Sidak Pal Singh Aurelien Lucchi AI4CE 113 0 0 04 Nov 2024
Remove Symmetries to Control Model Expressivity and Improve Optimization Liu Ziyin Yizhou Xu Isaac Chuang AAML 85 2 0 28 Aug 2024
Reparameterization invariance in approximate Bayesian inference Hrittik Roy M. Miani Carl Henrik Ek Philipp Hennig Marvin Pfortner Lukas Tatzel Søren Hauberg BDL 79 9 0 05 Jun 2024
The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective Nils Philipp Walter Linara Adilova Jilles Vreeken Michael Kamp AAML 97 2 0 27 May 2024
Flatness Improves Backbone Generalisation in Few-shot Classification Rui Li Martin Trapp Talal Alrawajfeh Arno Solin 102 0 0 11 Apr 2024
Information-Theoretic Generalization Bounds for Deep Neural Networks Haiyun He Christina Lee Yu 88 6 0 04 Apr 2024
Momentum-SAM: Sharpness Aware Minimization without Computational Overhead Marlon Becker Frederick Altrock Benjamin Risse 126 6 0 22 Jan 2024
Adapting Newton's Method to Neural Networks through a Summary of Higher-Order Derivatives Pierre Wolinski ODL 120 0 0 06 Dec 2023
Improving Multi-task Learning via Seeking Task-based Flat Regions Hoang Phan Lam C. Tran Ngoc N. Tran Nhat Ho Tuan Truong Qi Lei Nhat Ho Dinh Q. Phung Trung Le 203 11 0 24 Nov 2022
Extrapolation for Large-batch Training in Deep Learning Tao R. Lin Lingjing Kong Sebastian U. Stich Martin Jaggi 77 36 0 10 Jun 2020
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach Ryo Karakida S. Akaho S. Amari FedML 159 145 0 04 Jun 2018
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 178 799 0 24 May 2017
Fast Rates for Empirical Risk Minimization of Strict Saddle Problems Alon Gonen Shai Shalev-Shwartz 78 30 0 16 Jan 2017
An empirical analysis of the optimization of deep network loss surfaces Daniel Jiwoong Im Michael Tao K. Branson ODL 63 63 0 13 Dec 2016
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 91 236 0 22 Nov 2016
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 345 4,629 0 10 Nov 2016
A Convolutional Encoder Model for Neural Machine Translation Jonas Gehring Michael Auli David Grangier Yann N. Dauphin 80 450 0 07 Nov 2016
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys Pratik Chaudhari A. Choromańska Stefano Soatto Yann LeCun Carlo Baldassi C. Borgs J. Chayes Levent Sagun R. Zecchina ODL 96 774 0 06 Nov 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhiwen Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 903 6,796 0 26 Sep 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 427 2,941 0 15 Sep 2016
Wav2Letter: an End-to-End ConvNet-based Speech Recognition System R. Collobert Christian Puhrsch Gabriel Synnaeve 3DV 64 283 0 11 Sep 2016
On the Expressive Power of Deep Neural Networks M. Raghu Ben Poole Jon M. Kleinberg Surya Ganguli Jascha Narain Sohl-Dickstein 61 790 0 16 Jun 2016
Improving Variational Inference with Inverse Autoregressive Flow Diederik P. Kingma Tim Salimans Rafal Jozefowicz Xi Chen Ilya Sutskever Max Welling BDL DRL 142 1,823 0 15 Jun 2016
Optimization Methods for Large-Scale Machine Learning Léon Bottou Frank E. Curtis J. Nocedal 246 3,216 0 15 Jun 2016
Improved Techniques for Training GANs Tim Salimans Ian Goodfellow Wojciech Zaremba Vicki Cheung Alec Radford Xi Chen GAN 483 9,062 0 10 Jun 2016
Faster Eigenvector Computation via Shift-and-Invert Preconditioning Dan Garber Laurent Dinh Chi Jin Jascha Narain Sohl-Dickstein Samy Bengio Praneeth Netrapalli Aaron Sidford 275 3,702 0 26 May 2016
Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks Devansh Arpit Yingbo Zhou Bhargava U. Kota V. Govindaraju 65 127 0 04 Mar 2016
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks Tim Salimans Diederik P. Kingma ODL 194 1,944 0 25 Feb 2016
Autoencoding beyond pixels using a learned similarity metric Anders Boesen Lindbo Larsen Søren Kaae Sønderby Hugo Larochelle Ole Winther GAN 173 2,072 0 31 Dec 2015
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,322 0 10 Dec 2015
A note on the evaluation of generative models Lucas Theis Aaron van den Oord Matthias Bethge EGVM 131 1,145 0 05 Nov 2015
Understanding symmetries in deep networks Vijay Badrinarayanan Bamdev Mishra R. Cipolla 284 42 0 03 Nov 2015
Train faster, generalize better: Stability of stochastic gradient descent Moritz Hardt Benjamin Recht Y. Singer 116 1,242 0 03 Sep 2015
Listen, Attend and Spell William Chan Navdeep Jaitly Quoc V. Le Oriol Vinyals RALM 156 2,266 0 05 Aug 2015
Natural Neural Networks Guillaume Desjardins Karen Simonyan Razvan Pascanu Koray Kavukcuoglu 107 176 0 01 Jul 2015
Attention-Based Models for Speech Recognition J. Chorowski Dzmitry Bahdanau Dmitriy Serdyuk Kyunghyun Cho Yoshua Bengio 127 2,607 0 24 Jun 2015
Path-SGD: Path-Normalized Optimization in Deep Neural Networks Behnam Neyshabur Ruslan Salakhutdinov Nathan Srebro ODL 86 309 0 08 Jun 2015
Variational Inference with Normalizing Flows Danilo Jimenez Rezende S. Mohamed DRL BDL 318 4,197 0 21 May 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,328 0 11 Feb 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 326 18,647 0 06 Feb 2015
Explorations on high dimensional landscapes Levent Sagun V. U. Güney Gerard Ben Arous Yann LeCun 52 65 0 20 Dec 2014
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 280 19,107 0 20 Dec 2014
Deep Speech: Scaling up end-to-end speech recognition Awni Y. Hannun Carl Case Jared Casper Bryan Catanzaro G. Diamos ... R. Prenger S. Satheesh Shubho Sengupta Adam Coates A. Ng 183 2,128 0 17 Dec 2014
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 261 1,198 0 30 Nov 2014
NICE: Non-linear Independent Components Estimation Laurent Dinh David M. Krueger Yoshua Bengio DRL BDL 129 2,261 0 30 Oct 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 477 43,685 0 17 Sep 2014