On the Spectral Bias of Neural Networks

22 June 2018

Aaron Courville

Papers citing "On the Spectral Bias of Neural Networks"

50 / 269 papers shown

Title
Understanding Layer-wise Contributions in Deep Neural Networks through Spectral Analysis Yatin Dandi Arthur Jacot FAtt 21 4 0 06 Nov 2021
Solving Partial Differential Equations with Point Source Based on Physics-Informed Neural Networks Xiang Huang Hongsheng Liu Beiji Shi Zidong Wang Kan Yang ... Jing Zhou Fan Yu Bei Hua Lei Chen Bin Dong 11 20 0 02 Nov 2021
Dynamic Inference with Neural Interpreters Nasim Rahaman Muhammad Waleed Gondal S. Joshi Peter V. Gehler Yoshua Bengio Francesco Locatello Bernhard Schölkopf 34 31 0 12 Oct 2021
Mesh Draping: Parametrization-Free Neural Mesh Transfer Amir Hertz Or Perel Raja Giryes O. Sorkine-Hornung Daniel Cohen-Or 3DH AI4CE 19 4 0 11 Oct 2021
3D Infomax improves GNNs for Molecular Property Prediction Hannes Stärk Dominique Beaini Gabriele Corso Prudencio Tossou Christian Dallago Stephan Günnemann Pietro Lió AI4CE 36 203 0 08 Oct 2021
Multi-Head ReLU Implicit Neural Representation Networks Arya Aftab Alireza Morsali 42 11 0 07 Oct 2021
Spectral Bias in Practice: The Role of Function Frequency in Generalization Sara Fridovich-Keil Raphael Gontijo-Lopes Rebecca Roelofs 41 28 0 06 Oct 2021
Deep Neural Networks and Tabular Data: A Survey V. Borisov Tobias Leemann Kathrin Seßler Johannes Haug Martin Pawelczyk Gjergji Kasneci LMTD 27 647 0 05 Oct 2021
Improved architectures and training algorithms for deep operator networks Sizhuang He Hanwen Wang P. Perdikaris AI4CE 52 105 0 04 Oct 2021
Neural Knitworks: Patched Neural Implicit Representation Networks Mikolaj Czerkawski Javier Cardona Robert C. Atkinson W. Michie I. Andonovic C. Clemente Christos Tachtatzis 36 11 0 29 Sep 2021
ImplicitVol: Sensorless 3D Ultrasound Reconstruction with Deep Implicit Representation P. Yeung L. Hesse M. Aliasi Monique C. Haak the INTERGROWTH-21st Consortium Weidi Xie A. Namburete 20 20 0 24 Sep 2021
Learning in Sinusoidal Spaces with Physics-Informed Neural Networks Jian Cheng Wong C. Ooi Abhishek Gupta Yew-Soon Ong AI4CE PINN SSL 18 76 0 20 Sep 2021
Multiple shooting for training neural differential equations on time series Evren Mert Turan J. Jäschke AI4TS 40 23 0 14 Sep 2021
MLReal: Bridging the gap between training on synthetic data and real data applications in machine learning T. Alkhalifah Hanchen Wang O. Ovcharenko OOD 47 65 0 11 Sep 2021
imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Pose Thiemo Alldieck Hongyi Xu C. Sminchisescu 3DH 43 116 0 24 Aug 2021
Data-informed Deep Optimization Lulu Zhang Z. Xu Yaoyu Zhang AI4CE 32 3 0 17 Jul 2021
Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations Benjamin Moseley Andrew Markham T. Nissen‐Meyer PINN 48 209 0 16 Jul 2021
On Measuring and Controlling the Spectral Bias of the Deep Image Prior Prithvijit Chakrabarty Pascal Mettes Subhransu Maji Cees G. M. Snoek 13 60 0 02 Jul 2021
Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks S. Maddu D. Sturm Christian L. Müller I. Sbalzarini AI4CE 23 81 0 02 Jul 2021
Structure First Detail Next: Image Inpainting with Pyramid Generator Shuyi Qu Zhenxing Niu Kaizhu Huang Jianke Zhu M. Protter Gadi Zimerman Yinghui Xu 15 4 0 16 Jun 2021
Long-time integration of parametric evolution equations with physics-informed DeepONets Sizhuang He P. Perdikaris AI4CE 24 117 0 09 Jun 2021
Spline Positional Encoding for Learning 3D Implicit Signed Distance Fields Peng-Shuai Wang Yang Liu Yu-Qi Yang Xin Tong 3DPC 33 28 0 03 Jun 2021
A neural anisotropic view of underspecification in deep learning Guillermo Ortiz-Jiménez I. Salazar-Reque Apostolos Modas Seyed-Mohsen Moosavi-Dezfooli P. Frossard 34 6 0 29 Apr 2021
SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimization Amir Hertz Or Perel Raja Giryes O. Sorkine-Hornung Daniel Cohen-Or 28 67 0 19 Apr 2021
Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks N. Sukumar Ankit Srivastava PINN AI4CE 41 241 0 17 Apr 2021
COTR: Correspondence Transformer for Matching Across Images Wei Jiang Eduard Trulls J. Hosang Andrea Tagliasacchi K. M. Yi ViT 46 258 0 25 Mar 2021
The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation Q. Du Yiqi Gu Haizhao Yang Chao Zhou 26 20 0 21 Mar 2021
Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses Jing Jin Mantang Guo Junhui Hou Hui Liu H. Xiong 16 6 0 14 Feb 2021
Fighting deepfakes by detecting GAN DCT anomalies O. Giudice Luca Guarnera Sebastiano Battiato 28 86 0 24 Jan 2021
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training Y. Fu Haoran You Yang Katie Zhao Yue Wang Chaojian Li K. Gopalakrishnan Zhangyang Wang Yingyan Lin MQ 35 32 0 24 Dec 2020
On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks Sizhuang He Hanwen Wang P. Perdikaris 131 438 0 18 Dec 2020
Object-Centric Neural Scene Rendering Michelle Guo Alireza Fathi Jiajun Wu Thomas Funkhouser 30 104 0 15 Dec 2020
Positional Encoding as Spatial Inductive Bias in GANs Rui Xu Xintao Wang Kai-xiang Chen Bolei Zhou Chen Change Loy GAN 27 89 0 09 Dec 2020
Fourier-domain Variational Formulation and Its Well-posedness for Supervised Learning Tao Luo Zheng Ma Zhiwei Wang Zhi-Qin John Xu Yaoyu Zhang OOD 44 4 0 06 Dec 2020
Unsupervised part representation by Flow Capsules S. Sabour Andrea Tagliasacchi S. Yazdani Geoffrey E. Hinton David J. Fleet 3DPC OCL 30 37 0 27 Nov 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 50 257 0 18 Nov 2020
Predicting Infectiousness for Proactive Contact Tracing Yoshua Bengio Prateek Gupta Tegan Maharaj Nasim Rahaman Martin Weiss ... Irina Rish Bernhard Schölkopf Abhinav Sharma Jian Tang Andrew Robert Williams 37 15 0 23 Oct 2020
Spatial Frequency Bias in Convolutional Generative Adversarial Networks Mahyar Khayatkhoei Ahmed Elgammal GAN 18 45 0 04 Oct 2020
Dodging DeepFake Detection via Implicit Spatial-Domain Notch Filtering Yihao Huang Felix Juefei Xu Qing-Wu Guo Yang Liu G. Pu 33 20 0 19 Sep 2020
Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization Neha S. Wadia Daniel Duckworth S. Schoenholz Ethan Dyer Jascha Narain Sohl-Dickstein 27 13 0 17 Aug 2020
Neural Reflectance Fields for Appearance Acquisition Sai Bi Zexiang Xu Pratul P. Srinivasan B. Mildenhall Kalyan Sunkavalli Milovs Havsan Yannick Hold-Geoffroy D. Kriegman R. Ramamoorthi 3DH 15 239 0 09 Aug 2020
When and why PINNs fail to train: A neural tangent kernel perspective Sizhuang He Xinling Yu P. Perdikaris 33 878 0 28 Jul 2020
Explicit Regularisation in Gaussian Noise Injections A. Camuto M. Willetts Umut Simsekli Stephen J. Roberts Chris Holmes 23 55 0 14 Jul 2020
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis Katja Schwarz Yiyi Liao Michael Niemeyer Andreas Geiger VGen 75 864 0 05 Jul 2020
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains Matthew Tancik Pratul P. Srinivasan B. Mildenhall Sara Fridovich-Keil N. Raghavan Utkarsh Singhal R. Ramamoorthi Jonathan T. Barron Ren Ng 60 2,344 0 18 Jun 2020
Neural Anisotropy Directions Guillermo Ortiz-Jiménez Apostolos Modas Seyed-Mohsen Moosavi-Dezfooli P. Frossard 31 16 0 17 Jun 2020
On the training dynamics of deep networks with $L_2$ regularization Aitor Lewkowycz Guy Gur-Ari 36 53 0 15 Jun 2020
Towards Frequency-Based Explanation for Robust CNN Zifan Wang Yilin Yang Ankit Shrivastava Varun Rawal Zihao Ding AAML FAtt 13 47 0 06 May 2020
Neural Additive Models: Interpretable Machine Learning with Neural Nets Rishabh Agarwal Levi Melnick Nicholas Frosst Xuezhou Zhang Ben Lengerich R. Caruana Geoffrey E. Hinton 46 405 0 29 Apr 2020
Predicting the outputs of finite deep neural networks trained with noisy gradients Gadi Naveh Oded Ben-David H. Sompolinsky Z. Ringel 11 20 0 02 Apr 2020