Weight Expansion: A New Perspective on Dropout and Generalization

23 January 2022

Papers citing "Weight Expansion: A New Perspective on Dropout and Generalization"

25 / 25 papers shown

Title
Enhancing Robust Fairness via Confusional Spectral Regularization Gaojie Jin Sihao Wu Jiaxu Liu Tianjin Huang Ronghui Mu 154 1 0 22 Jan 2025
Enhancing Adversarial Training with Second-Order Statistics of Weights Gao Jin Xinping Yi Wei Huang S. Schewe Xiaowei Huang AAML 61 47 0 11 Mar 2022
Representation Based Complexity Measures for Predicting Generalization in Deep Learning Parth Natekar Manik Sharma 35 36 0 04 Dec 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 176 1,323 0 03 Oct 2020
Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms Mahdi Haghifam Jeffrey Negrea Ashish Khisti Daniel M. Roy Gintare Karolina Dziugaite 112 105 0 27 Apr 2020
Rethinking Bias-Variance Trade-off for Generalization of Neural Networks Zitong Yang Yaodong Yu Chong You Jacob Steinhardt Yi-An Ma 65 183 0 26 Feb 2020
Improving Generalization by Controlling Label-Noise Information in Neural Network Weights Hrayr Harutyunyan Kyle Reing Greg Ver Steeg Aram Galstyan NoLa 30 55 0 19 Feb 2020
Fantastic Generalization Measures and Where to Find Them Yiding Jiang Behnam Neyshabur H. Mobahi Dilip Krishnan Samy Bengio AI4CE 99 599 0 04 Dec 2019
Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates Jeffrey Negrea Mahdi Haghifam Gintare Karolina Dziugaite Ashish Khisti Daniel M. Roy FedML 148 149 0 06 Nov 2019
Reducing Transformer Depth on Demand with Structured Dropout Angela Fan Edouard Grave Armand Joulin 98 588 0 25 Sep 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 68 500 0 31 May 2019
Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks Gaël Letarte Pascal Germain Benjamin Guedj Franccois Laviolette MQ AI4CE UQCV 63 54 0 24 May 2019
Generalization in Deep Networks: The Role of Distance from Initialization Vaishnavh Nagarajan J. Zico Kolter ODL 69 96 0 07 Jan 2019
Improving Generalization Performance by Switching from Adam to SGD N. Keskar R. Socher ODL 72 522 0 20 Dec 2017
Three Factors Influencing Minima in SGD Stanislaw Jastrzebski Zachary Kenton Devansh Arpit Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey 74 459 0 13 Nov 2017
Information-theoretic analysis of generalization capability of learning algorithms Aolin Xu Maxim Raginsky 127 442 0 22 May 2017
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys Pratik Chaudhari A. Choromańska Stefano Soatto Yann LeCun Carlo Baldassi C. Borgs J. Chayes Levent Sagun R. Zecchina ODL 92 769 0 06 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 384 2,922 0 15 Sep 2016
Enriching Word Vectors with Subword Information Piotr Bojanowski Edouard Grave Armand Joulin Tomas Mikolov NAI SSL VLM 200 9,944 0 15 Jul 2016
Variational Dropout and the Local Reparameterization Trick Diederik P. Kingma Tim Salimans Max Welling BDL 187 1,500 0 08 Jun 2015
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 255 583 0 27 Feb 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 143 1,830 0 20 Dec 2013
A PAC-Bayesian Tutorial with A Dropout Bound David A. McAllester 72 140 0 08 Jul 2013
Dropout Training as Adaptive Regularization Stefan Wager Sida I. Wang Percy Liang 112 597 0 04 Jul 2013
Improving neural networks by preventing co-adaptation of feature detectors Geoffrey E. Hinton Nitish Srivastava A. Krizhevsky Ilya Sutskever Ruslan Salakhutdinov VLM 410 7,650 0 03 Jul 2012