Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum

26 July 2023

Papers citing "Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum"

27 / 27 papers shown

Title
Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations Kexuan Shi Hai Chen Leheng Zhang Shuhang Gu 81 1 0 17 Oct 2024
Generalization in Kernel Regression Under Realistic Assumptions Daniel Barzilai Ohad Shamir 76 17 0 26 Dec 2023
A Kernel Perspective of Skip Connections in Convolutional Networks Daniel Barzilai Amnon Geifman Meirav Galun Ronen Basri 55 13 0 27 Nov 2022
A Fast, Well-Founded Approximation to the Empirical Neural Tangent Kernel Mohamad Amin Mohamadi Wonho Bae Danica J. Sutherland AAML 75 28 0 25 Jun 2022
Fast Finite Width Neural Tangent Kernel Roman Novak Jascha Narain Sohl-Dickstein S. Schoenholz AAML 53 55 0 17 Jun 2022
On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels Amnon Geifman Meirav Galun David Jacobs Ronen Basri 48 14 0 17 Mar 2022
Spectral Bias in Practice: The Role of Function Frequency in Generalization Sara Fridovich-Keil Raphael Gontijo-Lopes Rebecca Roelofs 70 30 0 06 Oct 2021
Deep Equals Shallow for ReLU Networks in Kernel Regimes A. Bietti Francis R. Bach 69 90 0 30 Sep 2020
Generalized Leverage Score Sampling for Neural Networks Jason D. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 62 43 0 21 Sep 2020
Finite Versus Infinite Neural Networks: an Empirical Study Jaehoon Lee S. Schoenholz Jeffrey Pennington Ben Adlam Lechao Xiao Roman Novak Jascha Narain Sohl-Dickstein 75 214 0 31 Jul 2020
When and why PINNs fail to train: A neural tangent kernel perspective Sizhuang He Xinling Yu P. Perdikaris 135 914 0 28 Jul 2020
Tensor Programs II: Neural Tangent Kernel for Any Architecture Greg Yang 114 136 0 25 Jun 2020
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains Matthew Tancik Pratul P. Srinivasan B. Mildenhall Sara Fridovich-Keil N. Raghavan Utkarsh Singhal R. Ramamoorthi Jonathan T. Barron Ren Ng 124 2,421 0 18 Jun 2020
Frequency Bias in Neural Networks for Input of Non-Uniform Density Ronen Basri Meirav Galun Amnon Geifman David Jacobs Yoni Kasten S. Kritchman 80 184 0 10 Mar 2020
Why Do Deep Residual Networks Generalize Better than Deep Feedforward Networks? -- A Neural Tangent Kernel Perspective Kaixuan Huang Yuqing Wang Molei Tao T. Zhao MLT 51 97 0 14 Feb 2020
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 98 219 0 03 Dec 2019
Enhanced Convolutional Neural Tangent Kernels Zhiyuan Li Ruosong Wang Dingli Yu S. Du Wei Hu Ruslan Salakhutdinov Sanjeev Arora 64 132 0 03 Nov 2019
A Fine-Grained Spectral Perspective on Neural Networks Greg Yang Hadi Salman 76 113 0 24 Jul 2019
Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound Zhao Song Xin Yang 60 91 0 09 Jun 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 63 57 0 28 May 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks Guodong Zhang James Martens Roger C. Grosse ODL 88 124 0 27 May 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 230 996 0 01 Apr 2019
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation Greg Yang 148 287 0 13 Feb 2019
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks Zhi-Qin John Xu Yaoyu Zhang Yaoyu Zhang Yan Xiao Zheng Ma 124 516 0 19 Jan 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 835 0 19 Dec 2018
The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning Siyuan Ma Raef Bassily M. Belkin 79 288 0 18 Dec 2017
Large Batch Training of Convolutional Networks Yang You Igor Gitman Boris Ginsburg ODL 128 848 0 13 Aug 2017