On Lazy Training in Differentiable Programming

19 December 2018

Papers citing "On Lazy Training in Differentiable Programming"

50 / 246 papers shown

Title
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization) Zhenyu Zhu Fanghui Liu Grigorios G. Chrysos V. Cevher 39 19 0 15 Sep 2022
Differentiable Programming for Earth System Modeling Maximilian Gelbrecht Alistair J R White S. Bathiany Niklas Boers 21 16 0 29 Aug 2022
Gradient descent provably escapes saddle points in the training of shallow ReLU networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 36 5 0 03 Aug 2022
Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability Z. Li Zixuan Wang Jian Li 19 44 0 26 Jul 2022
The Neural Race Reduction: Dynamics of Abstraction in Gated Networks Andrew M. Saxe Shagun Sodhani Sam Lewallen AI4CE 32 34 0 21 Jul 2022
Graph Neural Network Bandits Parnian Kassraie Andreas Krause Ilija Bogunovic 26 11 0 13 Jul 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 45 28 0 08 Jul 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 25 114 0 30 Jun 2022
Learning sparse features can lead to overfitting in neural networks Leonardo Petrini Francesco Cagnetta Eric Vanden-Eijnden M. Wyart MLT 42 23 0 24 Jun 2022
Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation Loucas Pillaud-Vivien J. Reygner Nicolas Flammarion NoLa 33 31 0 20 Jun 2022
Wide Bayesian neural networks have a simple weight posterior: theory and accelerated sampling Jiri Hron Roman Novak Jeffrey Pennington Jascha Narain Sohl-Dickstein UQCV BDL 48 6 0 15 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 45 71 0 14 Jun 2022
Overcoming the Spectral Bias of Neural Value Approximation Ge Yang Anurag Ajay Pulkit Agrawal 34 25 0 09 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 29 10 0 08 Jun 2022
Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow Adam Subel Yifei Guan Ashesh Chattopadhyay Pedram Hassanzadeh AI4CE 35 41 0 07 Jun 2022
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs Etienne Boursier Loucas Pillaud-Vivien Nicolas Flammarion ODL 27 58 0 02 Jun 2022
Analyzing Tree Architectures in Ensembles via Neural Tangent Kernel Ryuichi Kanoh M. Sugiyama 31 2 0 25 May 2022
One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks Shutong Wu Sizhe Chen Cihang Xie X. Huang AAML 51 27 0 24 May 2022
Transition to Linearity of General Neural Networks with Directed Acyclic Graph Architecture Libin Zhu Chaoyue Liu M. Belkin GNN AI4CE 23 4 0 24 May 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon Cengiz Pehlevan MLT 40 77 0 19 May 2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 59 23 0 18 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 42 121 0 03 May 2022
Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes Chao Ma D. Kunin Lei Wu Lexing Ying 25 27 0 24 Apr 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 36 13 0 22 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Random matrix analysis of deep neural network weight matrices M. Thamm Max Staats B. Rosenow 37 12 0 28 Mar 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 39 13 0 22 Mar 2022
Robust Training under Label Noise by Over-parameterization Sheng Liu Zhihui Zhu Qing Qu Chong You NoLa OOD 32 106 0 28 Feb 2022
On the Benefits of Large Learning Rates for Kernel Methods Gaspard Beugnot Julien Mairal Alessandro Rudi 27 11 0 28 Feb 2022
The Spectral Bias of Polynomial Neural Networks Moulik Choraria L. Dadi Grigorios G. Chrysos Julien Mairal V. Cevher 24 18 0 27 Feb 2022
A Geometric Understanding of Natural Gradient Qinxun Bai S. Rosenberg Wei Xu 21 2 0 13 Feb 2022
Tight Convergence Rate Bounds for Optimization Under Power Law Spectral Conditions Maksim Velikanov Dmitry Yarotsky 11 6 0 02 Feb 2022
Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks R. Veiga Ludovic Stephan Bruno Loureiro Florent Krzakala Lenka Zdeborová MLT 24 31 0 01 Feb 2022
Stochastic Neural Networks with Infinite Width are Deterministic Liu Ziyin Hanlin Zhang Xiangming Meng Yuting Lu Eric P. Xing Masakuni Ueda 34 3 0 30 Jan 2022
Interplay between depth of neural networks and locality of target functions Takashi Mori Masakuni Ueda 25 0 0 28 Jan 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs Inbar Seroussi Gadi Naveh Zohar Ringel 35 51 0 31 Dec 2021
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 24 1 0 25 Dec 2021
Early Stopping for Deep Image Prior Hengkang Wang Taihui Li Zhong Zhuang Tiancong Chen Hengyue Liang Ju Sun 26 63 0 11 Dec 2021
SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning Yuege Xie Bobby Shi Hayden Schaeffer Rachel A. Ward 83 9 0 07 Dec 2021
Learning with convolution and pooling operations in kernel methods Theodor Misiakiewicz Song Mei MLT 15 29 0 16 Nov 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 25 18 0 11 Nov 2021
Understanding Layer-wise Contributions in Deep Neural Networks through Spectral Analysis Yatin Dandi Arthur Jacot FAtt 29 4 0 06 Nov 2021
Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks A. Shevchenko Vyacheslav Kungurtsev Marco Mondelli MLT 44 13 0 03 Nov 2021
Subquadratic Overparameterization for Shallow Neural Networks Chaehwan Song Ali Ramezani-Kebrya Thomas Pethick Armin Eftekhari V. Cevher 30 31 0 02 Nov 2021
Neural Networks as Kernel Learners: The Silent Alignment Effect Alexander B. Atanasov Blake Bordelon Cengiz Pehlevan MLT 26 75 0 29 Oct 2021
Does the Data Induce Capacity Control in Deep Learning? Rubing Yang Jialin Mao Pratik Chaudhari 35 15 0 27 Oct 2021
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion Zhemin Li Tao Sun Hongxia Wang Bao Wang 50 6 0 12 Oct 2021
Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective Adhyyan Narang Vidya Muthukumar A. Sahai SILM AAML 36 1 0 27 Sep 2021
Fast and Sample-Efficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaussian Moments Viktor Zaverkin David Holzmüller Ingo Steinwart Johannes Kastner 29 19 0 20 Sep 2021