Path-SGD: Path-Normalized Optimization in Deep Neural Networks

8 June 2015

Papers citing "Path-SGD: Path-Normalized Optimization in Deep Neural Networks"

50 / 83 papers shown

Title
The Empirical Impact of Reducing Symmetries on the Performance of Deep Ensembles and MoE Andrei Chernov Oleg Novitskij 48 0 0 24 Feb 2025
Application of Langevin Dynamics to Advance the Quantum Natural Gradient Optimization Algorithm Oleksandr Borysenko Mykhailo Bratchenko Ilya Lukin Mykola Luhanko Ihor Omelchenko Andrii Sotnikov Alessandro Lomi 55 0 0 17 Feb 2025
Monomial Matrix Group Equivariant Neural Functional Networks Hoang V. Tran Thieu N. Vo Tho H. Tran An T. Nguyen Tan M. Nguyen 54 5 0 18 Sep 2024
Do Sharpness-based Optimizers Improve Generalization in Medical Image Analysis? Mohamed Hassan Aleksandar Vakanski Min Xian AAML MedIm 41 1 0 07 Aug 2024
Hidden Synergy: $L_1$ Weight Normalization and 1-Path-Norm Regularization Aditya Biswas 41 0 0 29 Apr 2024
Understanding the Double Descent Phenomenon in Deep Learning Marc Lafon Alexandre Thomas 25 2 0 15 Mar 2024
Level Set Teleportation: An Optimization Perspective Aaron Mishkin A. Bietti Robert Mansel Gower 36 1 0 05 Mar 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 46 1 0 01 Mar 2024
Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures Paul Viallard Rémi Emonet Amaury Habrard Emilie Morvant Valentina Zantedeschi 39 3 0 19 Feb 2024
Improving Convergence and Generalization Using Parameter Symmetries Bo Zhao Robert Mansel Gower Robin Walters Rose Yu MoMe 33 13 0 22 May 2023
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization Agustinus Kristiadi Felix Dangel Philipp Hennig 32 11 0 14 Feb 2023
Equivariant Architectures for Learning in Deep Weight Spaces Aviv Navon Aviv Shamsian Idan Achituve Ethan Fetaya Gal Chechik Haggai Maron 49 63 0 30 Jan 2023
Quantifying the Impact of Label Noise on Federated Learning Shuqi Ke Chao Huang Xin Liu FedML 31 7 0 15 Nov 2022
Instance-Dependent Generalization Bounds via Optimal Transport Songyan Hou Parnian Kassraie Anastasis Kratsios Andreas Krause Jonas Rothfuss 26 6 0 02 Nov 2022
Symmetries, flat minima, and the conserved quantities of gradient flow Bo Zhao I. Ganev Robin Walters Rose Yu Nima Dehmamy 49 16 0 31 Oct 2022
Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel Sungyub Kim Si-hun Park Kyungsu Kim Eunho Yang BDL 32 4 0 30 Sep 2022
Local Identifiability of Deep ReLU Neural Networks: the Theory Joachim Bona-Pellissier Franccois Malgouyres François Bachoc FAtt 69 6 0 15 Jun 2022
Symmetry Teleportation for Accelerated Optimization B. Zhao Nima Dehmamy Robin Walters Rose Yu ODL 23 20 0 21 May 2022
A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima Tae-Eon Ko Xiantao Li 30 2 0 21 Mar 2022
Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning Yang Zhao Hao Zhang Xiuyuan Hu 38 116 0 08 Feb 2022
Weight Expansion: A New Perspective on Dropout and Generalization Gao Jin Xinping Yi Pengfei Yang Lijun Zhang S. Schewe Xiaowei Huang 29 5 0 23 Jan 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 27 34 0 20 Jan 2022
Measuring Complexity of Learning Schemes Using Hessian-Schatten Total Variation Shayan Aziznejad Joaquim Campos M. Unser 27 9 0 12 Dec 2021
GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophily Lun Du Xiaozhou Shi Qiang Fu Xiaojun Ma Hengyu Liu Shi Han Dongmei Zhang 40 105 0 29 Oct 2021
In Search of Probeable Generalization Measures Jonathan Jaegerman Khalil Damouni M. M. Ankaralı Konstantinos N. Plataniotis 27 2 0 23 Oct 2021
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks Tolga Ergen Mert Pilanci 34 16 0 18 Oct 2021
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 39 217 0 12 Oct 2021
Perturbated Gradients Updating within Unit Space for Deep Learning Ching-Hsun Tseng Liu Cheng Shin-Jye Lee Xiaojun Zeng 50 5 0 01 Oct 2021
Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks Rahul Parhi Robert D. Nowak 58 38 0 18 Sep 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 27 9 0 02 Aug 2021
An Embedding of ReLU Networks and an Analysis of their Identifiability Pierre Stock Rémi Gribonval 31 17 0 20 Jul 2021
What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory Rahul Parhi Robert D. Nowak MLT 38 70 0 07 May 2021
The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks Bohan Wang Qi Meng Wei Chen Tie-Yan Liu 30 33 0 11 Dec 2020
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics D. Kunin Javier Sagastuy-Breña Surya Ganguli Daniel L. K. Yamins Hidenori Tanaka 107 77 0 08 Dec 2020
The power of quantum neural networks Amira Abbas David Sutter Christa Zoufal Aurelien Lucchi Alessio Figalli Stefan Woerner 33 725 0 30 Oct 2020
Effective Regularization Through Loss-Function Metalearning Santiago Gonzalez Risto Miikkulainen 32 5 0 02 Oct 2020
Learning Optimal Representations with the Decodable Information Bottleneck Yann Dubois Douwe Kiela D. Schwab Ramakrishna Vedantam 28 43 0 27 Sep 2020
The Representation Theory of Neural Networks M. Armenta Pierre-Marc Jodoin 29 30 0 23 Jul 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei Jason D. Lee Tengyu Ma 32 94 0 15 Jun 2020
FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction Georgios Damaskinos R. Guerraoui Anne-Marie Kermarrec Vlad Nitu Rhicheek Patra Francois Taiani 21 54 0 12 Jun 2020
Dropout: Explicit Forms and Capacity Control R. Arora Peter L. Bartlett Poorya Mianjy Nathan Srebro 64 37 0 06 Mar 2020
On the distance between two neural networks and the stability of learning Jeremy Bernstein Arash Vahdat Yisong Yue Xuan Li ODL 200 57 0 09 Feb 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 27 168 0 19 Dec 2019
Information-Theoretic Local Minima Characterization and Regularization Zhiwei Jia Hao Su 27 19 0 19 Nov 2019
On Generalization Bounds of a Family of Recurrent Neural Networks Minshuo Chen Xingguo Li T. Zhao 19 70 0 28 Oct 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 52 324 0 13 Jun 2019
The Implicit Bias of AdaGrad on Separable Data Qian Qian Xiaoyuan Qian 37 23 0 09 Jun 2019
Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models Mor Shpigel Nacson Suriya Gunasekar Jason D. Lee Nathan Srebro Daniel Soudry 33 92 0 17 May 2019
A Priori Estimates of the Population Risk for Residual Networks E. Weinan Chao Ma Qingcan Wang UQCV 37 61 0 06 Mar 2019
A Theoretical Analysis of Deep Q-Learning Jianqing Fan Zhuoran Yang Yuchen Xie Zhaoran Wang 23 596 0 01 Jan 2019