Implicit Regularization in Matrix Factorization

25 May 2017

Srinadh Bhojanapalli

Papers citing "Implicit Regularization in Matrix Factorization"

50 / 111 papers shown

Title
Entropic Mirror Descent for Linear Systems: Polyak's Stepsize and Implicit Bias Yura Malitsky Alexander Posch 27 0 0 05 May 2025
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 59 0 0 11 Apr 2025
Forget the Data and Fine-Tuning! Just Fold the Network to Compress Dong Wang Haris Šikić Lothar Thiele O. Saukh 59 0 0 17 Feb 2025
Implicit Bias in Matrix Factorization and its Explicit Realization in a New Architecture Yikun Hou Suvrit Sra A. Yurtsever 34 0 0 28 Jan 2025
Optimization Insights into Deep Diagonal Linear Networks Hippolyte Labarrière C. Molinari Lorenzo Rosasco S. Villa Cristian Vega 76 0 0 21 Dec 2024
Multiple Rotation Averaging with Constrained Reweighting Deep Matrix Factorization Shiqi Li Jihua Zhu Yifan Xie Naiwen Hu Mingchen Zhu Zhongyu Li Di Wang 151 0 0 15 Sep 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 47 3 0 19 Aug 2024
How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD Pierfrancesco Beneventano Andrea Pinto Tomaso A. Poggio MLT 32 1 0 17 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 37 12 0 06 Jun 2024
Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion Zhiwei Bai Jiajie Zhao Yaoyu Zhang AI4CE 37 0 0 22 May 2024
$Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization$ Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization Shuo Xie Zhiyuan Li OffRL 47 13 0 05 Apr 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis Haupt ODL 44 3 0 12 Mar 2024
Learning Rich Rankings Arjun Seshadri Stephen Ragain J. Ugander 19 12 0 22 Dec 2023
In Search of a Data Transformation That Accelerates Neural Field Training Junwon Seo Sangyoon Lee Kwang In Kim Jaeho Lee 47 3 0 28 Nov 2023
Critical Learning Periods Emerge Even in Deep Linear Networks Michael Kleinman Alessandro Achille Stefano Soatto 41 3 0 23 Aug 2023
Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods A. Ma Yangchen Pan Amir-massoud Farahmand AAML 25 5 0 13 Aug 2023
Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization Hancheng Min Enrique Mallada René Vidal MLT 34 18 0 24 Jul 2023
Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization Kaiyue Wen Zhiyuan Li Tengyu Ma FAtt 38 26 0 20 Jul 2023
FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learning Chia-Hsiang Kao Yu-Chiang Frank Wang FedML 26 1 0 19 Jul 2023
The Interpolating Information Criterion for Overparameterized Models Liam Hodgkinson Christopher van der Heide Roberto Salomone Fred Roosta Michael W. Mahoney 20 7 0 15 Jul 2023
Efficiently Factorizing Boolean Matrices using Proximal Gradient Descent Sebastian Dalleiger Jilles Vreeken 16 5 0 14 Jul 2023
The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks Yuan Cao Difan Zou Yuan-Fang Li Quanquan Gu MLT 37 5 0 20 Jun 2023
ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Models Suzanna Parkinson Greg Ongie Rebecca Willett 65 6 0 24 May 2023
Collaborative Development of NLP models Fereshte Khani Marco Tulio Ribeiro 32 2 0 20 May 2023
Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stability Jingfeng Wu Vladimir Braverman Jason D. Lee 32 17 0 19 May 2023
Convergence of Alternating Gradient Descent for Matrix Factorization R. Ward T. Kolda 22 6 0 11 May 2023
Robust Implicit Regularization via Weight Normalization H. Chou Holger Rauhut Rachel A. Ward 33 7 0 09 May 2023
Saddle-to-Saddle Dynamics in Diagonal Linear Networks Scott Pesme Nicolas Flammarion 31 35 0 02 Apr 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 37 30 0 27 Mar 2023
Penalising the biases in norm regularisation enforces sparsity Etienne Boursier Nicolas Flammarion 37 14 0 02 Mar 2023
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels Simone Bombari Shayan Kiyani Marco Mondelli AAML 40 10 0 03 Feb 2023
Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent Avrajit Ghosh He Lyu Xitong Zhang Rongrong Wang 53 20 0 02 Feb 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 39 49 0 30 Jan 2023
Supervision Complexity and its Role in Knowledge Distillation Hrayr Harutyunyan A. S. Rawat A. Menon Seungyeon Kim Surinder Kumar 30 12 0 28 Jan 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 54 34 0 27 Jan 2023
Deep Linear Networks for Matrix Completion -- An Infinite Depth Limit Nadav Cohen Govind Menon Zsolt Veraszto ODL 29 7 0 22 Oct 2022
From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent Satyen Kale Jason D. Lee Chris De Sa Ayush Sekhari Karthik Sridharan 27 4 0 13 Oct 2022
Magnitude and Angle Dynamics in Training Single ReLU Neurons Sangmin Lee Byeongsu Sim Jong Chul Ye MLT 96 6 0 27 Sep 2022
A Validation Approach to Over-parameterized Matrix and Image Recovery Lijun Ding Zhen Qin Liwei Jiang Jinxin Zhou Zhihui Zhu 48 13 0 21 Sep 2022
Deep Linear Networks can Benignly Overfit when Shallow Ones Do Niladri S. Chatterji Philip M. Long 23 8 0 19 Sep 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 34 72 0 26 Aug 2022
Implicit Regularization with Polynomial Growth in Deep Tensor Factorization Kais Hariz Hachem Kadri Stéphane Ayache Maher Moakher Thierry Artières 26 2 0 18 Jul 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 42 27 0 08 Jul 2022
Orthogonal Matrix Retrieval with Spatial Consensus for 3D Unknown-View Tomography Shuai Huang Mona Zehni Ivan Dokmanić Zhizhen Zhao 23 12 0 06 Jul 2022
Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay Yuetian Luo Anru R. Zhang 29 19 0 17 Jun 2022
Benefits of Additive Noise in Composing Classes with Bounded Capacity A. F. Pour H. Ashtiani 33 3 0 14 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 40 70 0 14 Jun 2022
Efficient Scheduling of Data Augmentation for Deep Reinforcement Learning Byungchan Ko Jungseul Ok OnRL 27 5 0 01 Jun 2022
Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers Arda Sahiner Tolga Ergen Batu Mehmet Ozturkler John M. Pauly Morteza Mardani Mert Pilanci 40 33 0 17 May 2022
projUNN: efficient method for training deep networks with unitary matrices B. Kiani Randall Balestriero Yann LeCun S. Lloyd 43 32 0 10 Mar 2022