On Lazy Training in Differentiable Programming

19 December 2018

Papers citing "On Lazy Training in Differentiable Programming"

50 / 246 papers shown

Title
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks Zhichao Wang Yizhe Zhu 37 18 0 20 Sep 2021
Uniform Generalization Bounds for Overparameterized Neural Networks Sattar Vakili Michael Bromberg Jezabel R. Garcia Da-shan Shiu A. Bernacchia 25 19 0 13 Sep 2021
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 36 71 0 06 Sep 2021
Dash: Semi-Supervised Learning with Dynamic Thresholding Yi Tian Xu Lei Shang Jinxing Ye Qi Qian Yu-Feng Li Baigui Sun Hao Li Rong Jin 47 218 0 01 Sep 2021
Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization Difan Zou Yuan Cao Yuanzhi Li Quanquan Gu MLT AI4CE 47 39 0 25 Aug 2021
Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation Arnulf Jentzen Adrian Riekert 27 23 0 09 Jul 2021
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity Sebastian Lee Sebastian Goldt Andrew M. Saxe CLL 32 73 0 09 Jul 2021
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 42 75 0 28 Jun 2021
Locality defeats the curse of dimensionality in convolutional teacher-student scenarios Alessandro Favero Francesco Cagnetta M. Wyart 30 31 0 16 Jun 2021
A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs Gadi Naveh Zohar Ringel SSL MLT 36 31 0 08 Jun 2021
The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization Mufan Li Mihai Nica Daniel M. Roy 35 33 0 07 Jun 2021
Priors in Bayesian Deep Learning: A Review Vincent Fortuin UQCV BDL 33 124 0 14 May 2021
Global Convergence of Three-layer Neural Networks in the Mean Field Regime H. Pham Phan-Minh Nguyen MLT AI4CE 41 19 0 11 May 2021
Relative stability toward diffeomorphisms indicates performance in deep nets Leonardo Petrini Alessandro Favero Mario Geiger M. Wyart OOD 38 15 0 06 May 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 40 196 0 06 May 2021
RATT: Leveraging Unlabeled Data to Guarantee Generalization Saurabh Garg Sivaraman Balakrishnan J. Zico Kolter Zachary Chase Lipton 32 30 0 01 May 2021
Generalization Guarantees for Neural Architecture Search with Train-Validation Split Samet Oymak Mingchen Li Mahdi Soltanolkotabi AI4CE OOD 36 13 0 29 Apr 2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 172 28 0 22 Apr 2021
Understanding Overparameterization in Generative Adversarial Networks Yogesh Balaji M. Sajedi Neha Kalibhat Mucong Ding Dominik Stöger Mahdi Soltanolkotabi S. Feizi AI4CE 22 21 0 12 Apr 2021
Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions Patrick Cheridito Arnulf Jentzen Florian Rossmannek 24 10 0 19 Mar 2021
Computing the Information Content of Trained Neural Networks Jeremy Bernstein Yisong Yue 27 4 0 01 Mar 2021
Experiments with Rich Regime Training for Deep Learning Xinyan Li A. Banerjee 32 2 0 26 Feb 2021
Do Input Gradients Highlight Discriminative Features? Harshay Shah Prateek Jain Praneeth Netrapalli AAML FAtt 26 57 0 25 Feb 2021
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent Shahar Azulay E. Moroshko Mor Shpigel Nacson Blake E. Woodworth Nathan Srebro Amir Globerson Daniel Soudry AI4CE 35 73 0 19 Feb 2021
Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes Steffen Dereich Sebastian Kassing 34 27 0 16 Feb 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 27 250 0 12 Feb 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 76 45 0 04 Feb 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 130 168 0 29 Jan 2021
A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations Jianfeng Lu Yulong Lu Min Wang 36 37 0 05 Jan 2021
Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise Spencer Frei Yuan Cao Quanquan Gu FedML MLT 70 19 0 04 Jan 2021
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks Quynh N. Nguyen Marco Mondelli Guido Montúfar 25 81 0 21 Dec 2020
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 35 50 0 16 Dec 2020
A semigroup method for high dimensional committor functions based on neural network Haoya Li Y. Khoo Yinuo Ren Lexing Ying 16 6 0 12 Dec 2020
On 1/n neural representation and robustness Josue Nassar Piotr A. Sokól SueYeon Chung K. Harris Il Memming Park AAML OOD 24 23 0 08 Dec 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 50 258 0 18 Nov 2020
On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces Zhuoran Yang Chi Jin Zhaoran Wang Mengdi Wang Michael I. Jordan 39 18 0 09 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 77 671 0 06 Nov 2020
A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks Zhiqi Bu Shiyun Xu Kan Chen 33 17 0 25 Oct 2020
Stable ResNet Soufiane Hayou Eugenio Clerico Bo He George Deligiannidis Arnaud Doucet Judith Rousseau ODL SSeg 46 51 0 24 Oct 2020
Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regime Andrea Agazzi Jianfeng Lu 13 15 0 22 Oct 2020
A Unifying View on Implicit Bias in Training Linear Neural Networks Chulhee Yun Shankar Krishnan H. Mobahi MLT 18 80 0 06 Oct 2020
On the linearity of large non-linear models: when and why the tangent kernel is constant Chaoyue Liu Libin Zhu M. Belkin 21 140 0 02 Oct 2020
Deep Equals Shallow for ReLU Networks in Kernel Regimes A. Bietti Francis R. Bach 30 86 0 30 Sep 2020
How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks Keyulu Xu Mozhi Zhang Jingling Li S. Du Ken-ichi Kawarabayashi Stefanie Jegelka MLT 25 306 0 24 Sep 2020
Generalized Leverage Score Sampling for Neural Networks J. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 21 42 0 21 Sep 2020
Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy Zuyue Fu Zhuoran Yang Zhaoran Wang 21 42 0 02 Aug 2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 49 95 0 25 Jul 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo M. Wyart MLT 55 34 0 22 Jul 2020
Phase diagram for two-layer ReLU neural networks at infinite-width limit Tao Luo Zhi-Qin John Xu Zheng Ma Yaoyu Zhang 22 59 0 15 Jul 2020
Explicit Regularisation in Gaussian Noise Injections A. Camuto M. Willetts Umut Simsekli Stephen J. Roberts Chris Holmes 25 55 0 14 Jul 2020