The Power of Depth for Feedforward Neural Networks

12 December 2015

Papers citing "The Power of Depth for Feedforward Neural Networks"

50 / 367 papers shown

Title
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 44 13 0 11 May 2023
Multi-Path Transformer is Better: A Case Study on Neural Machine Translation Ye Lin Shuhan Zhou Yanyang Li Anxiang Ma Tong Xiao Jingbo Zhu 38 0 0 10 May 2023
Securing Neural Networks with Knapsack Optimization Yakir Gorski Amir Jevnisek S. Avidan AAML 34 0 0 20 Apr 2023
Deep neural network approximation of composite functions without the curse of dimensionality Adrian Riekert 26 0 0 12 Apr 2023
Depth Separation with Multilayer Mean-Field Networks Y. Ren Mo Zhou Rong Ge OOD 22 3 0 03 Apr 2023
Expressivity of Shallow and Deep Neural Networks for Polynomial Approximation Itai Shapira 11 0 0 06 Mar 2023
Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes Christian Haase Christoph Hertrich Georg Loho 34 22 0 24 Feb 2023
Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization Cameron Jakub Mihai Nica ODL 13 5 0 20 Feb 2023
Sharp Lower Bounds on Interpolation by Deep ReLU Neural Networks at Irregularly Spaced Data Jonathan W. Siegel 14 2 0 02 Feb 2023
LU decomposition and Toeplitz decomposition of a neural network Yucong Liu Simiao Jiao Lek-Heng Lim 30 7 0 25 Nov 2022
Multilayer Perceptron Network Discriminates Larval Zebrafish Genotype using Behaviour Christopher Fusco Angel G Allen 21 0 0 06 Nov 2022
A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 27 5 0 28 Oct 2022
On the Approximation and Complexity of Deep Neural Networks to Invariant Functions Gao Zhang Jin-Hui Wu Shao-Qun Zhang 16 0 0 27 Oct 2022
Non-Linear Coordination Graphs Yipeng Kang Tonghan Wang Xiao-Ren Wu Qianlan Yang Chongjie Zhang 37 9 0 26 Oct 2022
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work Jiawei Zhang Yushun Zhang Mingyi Hong Ruoyu Sun Zhi-Quan Luo 29 10 0 21 Oct 2022
Transformers Learn Shortcuts to Automata Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang OffRL LRM 48 156 0 19 Oct 2022
Improved Bounds on Neural Complexity for Representing Piecewise Linear Functions Kuan-Lin Chen H. Garudadri Bhaskar D. Rao 11 19 0 13 Oct 2022
On Scrambling Phenomena for Randomly Initialized Recurrent Networks Vaggos Chatziafratis Ioannis Panageas Clayton Sanford S. Stavroulakis 30 2 0 11 Oct 2022
Variational Inference for Infinitely Deep Neural Networks Achille Nazaret David M. Blei BDL 25 11 0 21 Sep 2022
Bootstrap Generalization Ability from Loss Landscape Perspective Huanran Chen Shitong Shao Ziyi Wang Zirui Shang Jin Chen Xiaofeng Ji Xinxiao Wu OOD 61 17 0 18 Sep 2022
Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality Stephan Wojtowytsch 28 1 0 02 Sep 2022
Delaunay-Triangulation-Based Learning with Hessian Total-Variation Regularization Mehrsa Pourya Alexis Goujon M. Unser 30 5 0 16 Aug 2022
Towards Antisymmetric Neural Ansatz Separation Aaron Zweig Joan Bruna 20 4 0 05 Aug 2022
On the Evaluation of User Privacy in Deep Neural Networks using Timing Side Channel Shubhi Shukla Manaar Alam Sarani Bhattacharya Debdeep Mukhopadhyay Pabitra Mitra AAML 27 2 0 01 Aug 2022
The BUTTER Zone: An Empirical Study of Training Dynamics in Fully Connected Neural Networks Charles Edison Tripp J. Perr-Sauer L. Hayne M. Lunacek Jamil Gafur AI4CE 28 0 0 25 Jul 2022
Blessing of Nonconvexity in Deep Linear Models: Depth Flattens the Optimization Landscape Around the True Solution Jianhao Ma S. Fattahi 44 5 0 15 Jul 2022
Size and depth of monotone neural networks: interpolation and approximation Dan Mikulincer Daniel Reichman 28 7 0 12 Jul 2022
Coin Flipping Neural Networks Yuval Sieradzki Nitzan Hodos Gal Yehuda Assaf Schuster UQCV 32 3 0 18 Jun 2022
On the Number of Regions of Piecewise Linear Neural Networks Alexis Goujon Arian Etemadi M. Unser 44 13 0 17 Jun 2022
$Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias$ Intrinsic dimensionality and generalization properties of the $\mathcal{R}$ -norm inductive bias Navid Ardeshir Daniel J. Hsu Clayton Sanford CML AI4CE 23 6 0 10 Jun 2022
Exponential Separations in Symmetric Neural Networks Aaron Zweig Joan Bruna 32 7 0 02 Jun 2022
Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks Zhiwei Bai Tao Luo Z. Xu Yaoyu Zhang 31 5 0 26 May 2022
CNNs Avoid Curse of Dimensionality by Learning on Patches Vamshi C. Madala S. Chandrasekaran Jason Bunk UQCV 33 5 0 22 May 2022
Convergence and Implicit Regularization Properties of Gradient Descent for Deep Residual Networks R. Cont Alain Rossier Renyuan Xu MLT 26 6 0 14 Apr 2022
$Training Fully Connected Neural Networks is $\exists\mathbb{R}$-Complete$ Training Fully Connected Neural Networks is $\exists\mathbb{R}$ -Complete Daniel Bertschinger Christoph Hertrich Paul Jungeblut Tillmann Miltzow Simon Weber OffRL 61 30 0 04 Apr 2022
Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection Jing-Xiao Liao Bo-Jian Hou Hangcheng Dong Hao Zhang Xiaoge Zhang Jinwei Sun Shiping Zhang Fenglei Fan 26 4 0 02 Apr 2022
Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation Josiah Park Stephan Wojtowytsch 26 1 0 25 Mar 2022
Towards understanding deep learning with the natural clustering prior Simon Carbonnelle 21 0 0 15 Mar 2022
Geometric Regularization from Overparameterization Nicholas J. Teague 22 1 0 18 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Width is Less Important than Depth in ReLU Neural Networks Gal Vardi Gilad Yehudai Ohad Shamir 3DV 21 9 0 08 Feb 2022
Training Thinner and Deeper Neural Networks: Jumpstart Regularization Carles Roger Riera Molina Camilo Rey Thiago Serra Eloi Puertas O. Pujol 27 4 0 30 Jan 2022
Interplay between depth of neural networks and locality of target functions Takashi Mori Masakuni Ueda 25 0 0 28 Jan 2022
Spherical Poisson Point Process Intensity Function Modeling and Estimation with Measure Transport T. L. J. Ng A. Zammit‐Mangion 24 3 0 24 Jan 2022
Depth and Feature Learning are Provably Beneficial for Neural Network Discriminators Carles Domingo-Enrich MLT MDE 31 0 0 27 Dec 2021
Integral representations of shallow neural network with Rectified Power Unit activation function Ahmed Abdeljawad Philipp Grohs 20 10 0 20 Dec 2021
A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation Wuyang Chen Xianzhi Du Fan Yang Lucas Beyer Xiaohua Zhai ... Huizhong Chen Jing Li Xiaodan Song Zhangyang Wang Denny Zhou ViT 29 20 0 17 Dec 2021
Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks Lechao Xiao 34 22 0 10 Dec 2021
Optimization-Based Separations for Neural Networks Itay Safran Jason D. Lee 182 14 0 04 Dec 2021
Error Bounds for a Matrix-Vector Product Approximation with Deep ReLU Neural Networks T. Getu 27 2 0 25 Nov 2021