Path-SGD: Path-Normalized Optimization in Deep Neural Networks

8 June 2015

Papers citing "Path-SGD: Path-Normalized Optimization in Deep Neural Networks"

34 / 84 papers shown

Title
A Theoretical Analysis of Deep Q-Learning Jianqing Fan Zhuoran Yang Yuchen Xie Zhaoran Wang 28 596 0 01 Jan 2019
Deep Frank-Wolfe For Neural Network Optimization Leonard Berrada Andrew Zisserman M. P. Kumar ODL 21 40 0 19 Nov 2018
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei Jason D. Lee Qiang Liu Tengyu Ma 28 245 0 12 Oct 2018
Capacity Control of ReLU Neural Networks by Basis-path Norm Shuxin Zheng Qi Meng Huishuai Zhang Wei-neng Chen Nenghai Yu Tie-Yan Liu 24 23 0 19 Sep 2018
Approximation and Estimation for High-Dimensional Deep Learning Networks Andrew R. Barron Jason M. Klusowski 27 59 0 10 Sep 2018
Deep Neural Networks with Multi-Branch Architectures Are Less Non-Convex Hongyang R. Zhang Junru Shao Ruslan Salakhutdinov 39 14 0 06 Jun 2018
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced S. Du Wei Hu Jason D. Lee MLT 42 237 0 04 Jun 2018
The Singular Values of Convolutional Layers Hanie Sedghi Vineet Gupta Philip M. Long FAtt 48 200 0 26 May 2018
Decorrelated Batch Normalization Lei Huang Dawei Yang B. Lang Jia Deng 16 190 0 23 Apr 2018
Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision Sathya Ravi Tuan Dinh Vishnu Suresh Lokhande Vikas Singh AI4CE 33 22 0 17 Mar 2018
On the insufficiency of existing momentum schemes for Stochastic Optimization Rahul Kidambi Praneeth Netrapalli Prateek Jain Sham Kakade ODL 32 117 0 15 Mar 2018
Shampoo: Preconditioned Stochastic Tensor Optimization Vineet Gupta Tomer Koren Y. Singer ODL 22 201 0 26 Feb 2018
Characterizing Implicit Bias in Terms of Optimization Geometry Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro AI4CE 46 399 0 22 Feb 2018
Stronger generalization bounds for deep nets via a compression approach Sanjeev Arora Rong Ge Behnam Neyshabur Yi Zhang MLT AI4CE 41 631 0 14 Feb 2018
$$\mathcal{G}$-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space$ $\mathcal{G}$ -SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space Qi Meng Shuxin Zheng Huishuai Zhang Wei Chen Zhi-Ming Ma Tie-Yan Liu 35 38 0 11 Feb 2018
Riemannian approach to batch normalization Minhyung Cho Jaehyung Lee 29 94 0 27 Sep 2017
Orthogonal and Idempotent Transformations for Learning Deep Neural Networks Jingdong Wang Yajie Xing Kexin Zhang Cha Zhang 199 2 0 19 Jul 2017
Block-Normalized Gradient Method: An Empirical Study for Training Deep Neural Network Adams Wei Yu Lei Huang Qihang Lin Ruslan Salakhutdinov J. Carbonell ODL 16 24 0 16 Jul 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 110 1,239 0 27 Jun 2017
The Marginal Value of Adaptive Gradient Methods in Machine Learning Ashia Wilson Rebecca Roelofs Mitchell Stern Nathan Srebro Benjamin Recht ODL 20 1,013 0 23 May 2017
Geometry of Optimization and Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Ruslan Salakhutdinov Nathan Srebro AI4CE 19 132 0 08 May 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 51 283 0 26 Apr 2017
Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Gintare Karolina Dziugaite Daniel M. Roy 50 802 0 31 Mar 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 46 758 0 15 Mar 2017
Skip Connections Eliminate Singularities Emin Orhan Xaq Pitkow 36 25 0 31 Jan 2017
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 67 10,342 0 21 Jul 2016
Robust Large Margin Deep Neural Networks Jure Sokolić Raja Giryes Guillermo Sapiro M. Rodrigues 26 307 0 26 May 2016
FractalNet: Ultra-Deep Neural Networks without Residuals Gustav Larsson Michael Maire Gregory Shakhnarovich 45 933 0 24 May 2016
Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations Behnam Neyshabur Yuhuai Wu Ruslan Salakhutdinov Nathan Srebro AI4CE ODL 27 30 0 23 May 2016
Improved Dropout for Shallow and Deep Learning Zhe Li Boqing Gong Tianbao Yang BDL SyDa 30 79 0 06 Feb 2016
Data-Dependent Path Normalization in Neural Networks Behnam Neyshabur Ryota Tomioka Ruslan Salakhutdinov Nathan Srebro 32 22 0 20 Nov 2015
Why are deep nets reversible: A simple theory, with implications for training Sanjeev Arora Yingyu Liang Tengyu Ma 19 54 0 18 Nov 2015
Symmetry-invariant optimization in deep networks Vijay Badrinarayanan Bamdev Mishra R. Cipolla ODL 19 31 0 05 Nov 2015
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 127 577 0 27 Feb 2015