Towards Understanding the Spectral Bias of Deep Learning

3 December 2019

Quanquan Gu

Papers citing "Towards Understanding the Spectral Bias of Deep Learning"

29 / 129 papers shown

Title
Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networks Yuval Belfer Amnon Geifman Meirav Galun Ronen Basri 20 17 0 07 Apr 2021
The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation Q. Du Yiqi Gu Haizhao Yang Chao Zhou 26 20 0 21 Mar 2021
Exploring The Effect of High-frequency Components in GANs Training Ziqiang Li Pengfei Xia Xue Rui Bin Li GAN 46 17 0 20 Mar 2021
Double-descent curves in neural networks: a new perspective using Gaussian processes Ouns El Harzli Bernardo Cuenca Grau Guillermo Valle Pérez A. Louis 20 6 0 14 Feb 2021
Reproducing Activation Function for Deep Learning Senwei Liang Liyao Lyu Chunmei Wang Haizhao Yang 36 21 0 13 Jan 2021
Frequency Principle in Deep Learning Beyond Gradient-descent-based Training Yuheng Ma Zhi-Qin John Xu Jiwei Zhang 24 7 0 04 Jan 2021
On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks Sizhuang He Hanwen Wang P. Perdikaris 131 439 0 18 Dec 2020
Fourier-domain Variational Formulation and Its Well-posedness for Supervised Learning Tao Luo Zheng Ma Zhiwei Wang Zhi-Qin John Xu Yaoyu Zhang OOD 44 4 0 06 Dec 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 50 257 0 18 Nov 2020
On the exact computation of linear frequency principle dynamics and its generalization Tao Luo Zheng Ma Z. Xu Yaoyu Zhang 11 19 0 15 Oct 2020
Neural Thompson Sampling Weitong Zhang Dongruo Zhou Lihong Li Quanquan Gu 26 114 0 02 Oct 2020
Deep Equals Shallow for ReLU Networks in Kernel Regimes A. Bietti Francis R. Bach 28 86 0 30 Sep 2020
Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS Lin Chen Sheng Xu 30 93 0 22 Sep 2020
Implicit Regularization via Neural Feature Alignment A. Baratin Thomas George César Laurent R. Devon Hjelm Guillaume Lajoie Pascal Vincent Simon Lacoste-Julien 18 6 0 03 Aug 2020
When and why PINNs fail to train: A neural tangent kernel perspective Sizhuang He Xinling Yu P. Perdikaris 33 878 0 28 Jul 2020
Deep frequency principle towards understanding why deeper learning is faster Zhi-Qin John Xu Hanxu Zhou 16 44 0 28 Jul 2020
Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network Tianyang Hu Wei Cao Cong Lin Guang Cheng 19 51 0 06 Jul 2020
On the Similarity between the Laplace and Neural Tangent Kernels Amnon Geifman A. Yadav Yoni Kasten Meirav Galun David Jacobs Ronen Basri 13 88 0 03 Jul 2020
The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks Wei Hu Lechao Xiao Ben Adlam Jeffrey Pennington 23 62 0 25 Jun 2020
Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks Abdulkadir Canatar Blake Bordelon Cengiz Pehlevan 22 181 0 23 Jun 2020
Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime Atsushi Nitanda Taiji Suzuki 6 41 0 22 Jun 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 36 32 0 18 Jun 2020
Rethink the Connections among Generalization, Memorization and the Spectral Bias of DNNs Xiao Zhang Haoyi Xiong Dongrui Wu 9 12 0 29 Apr 2020
Frequency Bias in Neural Networks for Input of Non-Uniform Density Ronen Basri Meirav Galun Amnon Geifman David Jacobs Yoni Kasten S. Kritchman 39 183 0 10 Mar 2020
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks Zixiang Chen Yuan Cao Quanquan Gu Tong Zhang MLT 35 10 0 10 Feb 2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks Blake Bordelon Abdulkadir Canatar Cengiz Pehlevan 146 201 0 07 Feb 2020
Pyramid Convolutional RNN for MRI Image Reconstruction Eric Z. Chen Puyang Wang Xiao Chen Terrence Chen Shanhui Sun 13 41 0 02 Dec 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 8 762 0 26 Jun 2019
Exact Convergence Rates of the Neural Tangent Kernel in the Large Depth Limit Soufiane Hayou Arnaud Doucet Judith Rousseau 16 4 0 31 May 2019