Directional convergence and alignment in deep learning

11 June 2020

Papers citing "Directional convergence and alignment in deep learning"

37 / 37 papers shown

Title
Embedding principle of homogeneous neural network for classification problem Jiahan Zhang Tao Luo Yaoyu Zhang 9 0 0 18 May 2025
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 62 0 0 11 Apr 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 60 8 0 20 Feb 2025
The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networks Sholom Schechtman Nicolas Schreuder 185 0 0 08 Feb 2025
Grokking at the Edge of Numerical Stability Lucas Prieto Melih Barsbey Pedro A.M. Mediano Tolga Birdal 48 3 0 08 Jan 2025
Dynamics of Supervised and Reinforcement Learning in the Non-Linear Perceptron Christian Schmid James M. Murray 40 0 0 05 Sep 2024
Classifying Overlapping Gaussian Mixtures in High Dimensions: From Optimal Classifiers to Neural Nets Khen Cohen Noam Levi Yaron Oz BDL 31 1 0 28 May 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis Haupt ODL 44 3 0 12 Mar 2024
Implicit Bias and Fast Convergence Rates for Self-attention Bhavya Vasudeva Puneesh Deora Christos Thrampoulidis 37 13 0 08 Feb 2024
What's in a Prior? Learned Proximal Networks for Inverse Problems Zhenghan Fang Sam Buchanan Jeremias Sulam 31 11 0 22 Oct 2023
Fundamental Limits of Membership Inference Attacks on Machine Learning Models Eric Aubinais Elisabeth Gassiat Pablo Piantanida MIACV 50 2 0 20 Oct 2023
Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization Hancheng Min Enrique Mallada René Vidal MLT 34 19 0 24 Jul 2023
Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses G. Buzaglo Niv Haim Gilad Yehudai Gal Vardi Yakir Oz Yaniv Nikankin Michal Irani 34 10 0 04 Jul 2023
Fast Convergence in Learning Two-Layer Neural Networks with Separable Data Hossein Taheri Christos Thrampoulidis MLT 16 3 0 22 May 2023
ADS_UNet: A Nested UNet for Histopathology Image Segmentation Yi-Lun Yang S. Dasmahapatra S. Mahmoodi MedIm SSeg 26 16 0 10 Apr 2023
Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation Noel Loo Ramin Hasani Mathias Lechner Alexander Amini Daniela Rus DD 42 5 0 02 Feb 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 54 34 0 27 Jan 2023
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 167 68 0 27 Oct 2022
SGD with Large Step Sizes Learns Sparse Features Maksym Andriushchenko Aditya Varre Loucas Pillaud-Vivien Nicolas Flammarion 45 56 0 11 Oct 2022
Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions Arthur Jacot 36 25 0 29 Sep 2022
Importance Tempering: Group Robustness for Overparameterized Models Yiping Lu Wenlong Ji Zachary Izzo Lexing Ying 42 7 0 19 Sep 2022
On Generalization of Decentralized Learning with Separable Data Hossein Taheri Christos Thrampoulidis FedML 30 11 0 15 Sep 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 34 72 0 26 Aug 2022
Imbalance Trouble: Revisiting Neural-Collapse Geometry Christos Thrampoulidis Ganesh Ramachandra Kini V. Vakilian Tina Behnia 26 69 0 10 Aug 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 42 27 0 08 Jul 2022
Warped Convolutional Networks: Bridge Homography to sl(3) algebra by Group Convolution Xinrui Zhan Yang Li Wenyu Liu Jianke Zhu 30 0 0 23 Jun 2022
Reconstructing Training Data from Trained Neural Networks Niv Haim Gal Vardi Gilad Yehudai Ohad Shamir Michal Irani 40 132 0 15 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 40 70 0 14 Jun 2022
Adversarial Reprogramming Revisited Matthias Englert R. Lazic AAML 29 8 0 07 Jun 2022
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs Etienne Boursier Loucas Pillaud-Vivien Nicolas Flammarion ODL 24 58 0 02 Jun 2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 59 23 0 18 May 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 46 29 0 27 Jan 2022
On Margin Maximization in Linear and ReLU Networks Gal Vardi Ohad Shamir Nathan Srebro 50 28 0 06 Oct 2021
Perturbated Gradients Updating within Unit Space for Deep Learning Ching-Hsun Tseng Liu Cheng Shin-Jye Lee Xiaojun Zeng 45 5 0 01 Oct 2021
The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks Bohan Wang Qi Meng Wei Chen Tie-Yan Liu 30 33 0 11 Dec 2020
Deep Networks and the Multiple Manifold Problem Sam Buchanan D. Gilboa John N. Wright 166 39 0 25 Aug 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms Noam Razin Nadav Cohen 24 155 0 13 May 2020