Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks

27 January 2022

Papers citing "Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks"

27 / 27 papers shown

Title
Tensor-GaLore: Memory-Efficient Training via Gradient Tensor Decomposition Robert Joseph George David Pitt Jiawei Zhao Jean Kossaifi Cheng Luo Yuandong Tian Anima Anandkumar 33 1 0 04 Jan 2025
Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning Nadav Cohen Noam Razin 31 0 0 25 Aug 2024
Algorithmic Regularization in Tensor Optimization: Towards a Lifted Approach in Matrix Sensing Ziye Ma Javad Lavaei Somayeh Sojoudi 28 2 0 24 Oct 2023
A Quadratic Synchronization Rule for Distributed Deep Learning Xinran Gu Kaifeng Lyu Sanjeev Arora Jingzhao Zhang Longbo Huang 51 1 0 22 Oct 2023
Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization Hancheng Min Enrique Mallada René Vidal MLT 32 19 0 24 Jul 2023
Implicit regularization in AI meets generalized hardness of approximation in optimization -- Sharp results for diagonal linear networks J. S. Wind Vegard Antun A. Hansen 17 4 0 13 Jul 2023
Robust Sparse Mean Estimation via Incremental Learning Jianhao Ma Ruidi Chen Yinghui He S. Fattahi Wei Hu 36 0 0 24 May 2023
Theoretical Analysis of Inductive Biases in Deep Convolutional Networks Zihao Wang Lei Wu 23 19 0 15 May 2023
Robust Implicit Regularization via Weight Normalization H. Chou Holger Rauhut Rachel A. Ward 28 7 0 09 May 2023
What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement Yotam Alexander Nimrod De La Vega Noam Razin Nadav Cohen 23 4 0 20 Mar 2023
Implicit Regularization Leads to Benign Overfitting for Sparse Linear Regression Mo Zhou Rong Ge 27 2 0 01 Feb 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 48 34 0 27 Jan 2023
On the Ability of Graph Neural Networks to Model Interactions Between Vertices Noam Razin Tom Verbin Nadav Cohen 19 10 0 29 Nov 2022
Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets Edo Cohen-Karlik Itamar Menuhin-Gruman Raja Giryes Nadav Cohen Amir Globerson 23 4 0 25 Oct 2022
Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition Jianhao Ma Li-Zhen Guo S. Fattahi 38 4 0 01 Oct 2022
Incremental Learning in Diagonal Linear Networks Raphael Berthier CLL AI4CE 27 16 0 31 Aug 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 34 72 0 26 Aug 2022
Implicit Regularization with Polynomial Growth in Deep Tensor Factorization Kais Hariz Hachem Kadri Stéphane Ayache Maher Moakher Thierry Artières 26 2 0 18 Jul 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 34 27 0 08 Jul 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 37 69 0 14 Jun 2022
Permutation Search of Tensor Network Structures via Local Sampling C. Li Junhua Zeng Zerui Tao Qianchuan Zhao 18 19 0 14 Jun 2022
More is Less: Inducing Sparsity via Overparameterization H. Chou J. Maly Holger Rauhut 30 25 0 21 Dec 2021
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 88 98 0 13 Oct 2021
On Margin Maximization in Linear and ReLU Networks Gal Vardi Ohad Shamir Nathan Srebro 47 28 0 06 Oct 2021
Continuous vs. Discrete Optimization of Deep Neural Networks Omer Elkabetz Nadav Cohen 62 44 0 14 Jul 2021
Implicit Regularization in Deep Tensor Factorization P. Milanesi Hachem Kadri Stéphane Ayache Thierry Artières 44 9 0 04 May 2021
Towards Understanding Learning in Neural Networks with Linear Teachers Roei Sarussi Alon Brutzkus Amir Globerson FedML MLT 55 20 0 07 Jan 2021