A Convergence Theory for Deep Learning via Over-Parameterization

9 November 2018

Papers citing "A Convergence Theory for Deep Learning via Over-Parameterization"

50 / 354 papers shown

Title
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 39 13 0 22 Mar 2022
TVConv: Efficient Translation Variant Convolution for Layout-aware Visual Processing Jie Chen Tianlang He Weipeng Zhuo Li Ma Sangtae Ha Shueng-Han Gary Chan CVBM 21 24 0 20 Mar 2022
On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels Amnon Geifman Meirav Galun David Jacobs Ronen Basri 30 13 0 17 Mar 2022
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning Haoxiang Wang Yite Wang Ruoyu Sun Bo-wen Li 31 27 0 17 Mar 2022
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning Weixin Liang Yuhui Zhang Yongchan Kwon Serena Yeung James Zou VLM 50 394 0 03 Mar 2022
The Spectral Bias of Polynomial Neural Networks Moulik Choraria L. Dadi Grigorios G. Chrysos Julien Mairal V. Cevher 24 18 0 27 Feb 2022
Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity Shiyun Xu Zhiqi Bu Pratik Chaudhari Ian Barnett 27 21 0 25 Feb 2022
Semi-Implicit Hybrid Gradient Methods with Application to Adversarial Robustness Beomsu Kim Junghoon Seo AAML 22 0 0 21 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Benchmark Assessment for DeepSpeed Optimization Library G. Liang I. Alsmadi 34 3 0 12 Feb 2022
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 38 1 0 07 Feb 2022
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning Wei Huang Chunrui Liu Yilan Chen Tianyu Liu R. Xu BDL MLT 19 2 0 04 Feb 2022
Neural Collaborative Filtering Bandits via Meta Learning Yikun Ban Yunzhe Qi Tianxin Wei Jingrui He OffRL 33 9 0 31 Jan 2022
Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks Bartlomiej Polaczyk J. Cyranka ODL 33 3 0 28 Jan 2022
Learning Proximal Operators to Discover Multiple Optima Lingxiao Li Noam Aigerman Vladimir G. Kim Jiajin Li Kristjan Greenewald Mikhail Yurochkin Justin Solomon 47 1 0 28 Jan 2022
A Kernel-Expanded Stochastic Neural Network Y. Sun F. Liang 23 5 0 14 Jan 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
In Defense of the Unitary Scalarization for Deep Multi-Task Learning Vitaly Kurin Alessandro De Palma Ilya Kostrikov Shimon Whiteson M. P. Kumar 39 74 0 11 Jan 2022
Stochastic Weight Averaging Revisited Hao Guo Jiyong Jin B. Liu 35 29 0 03 Jan 2022
Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs Inbar Seroussi Gadi Naveh Zohar Ringel 35 51 0 31 Dec 2021
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 24 1 0 25 Dec 2021
On the Convergence and Robustness of Adversarial Training Yisen Wang Xingjun Ma James Bailey Jinfeng Yi Bowen Zhou Quanquan Gu AAML 212 345 0 15 Dec 2021
Rethinking Influence Functions of Neural Networks in the Over-parameterized Regime Rui Zhang Shihua Zhang TDI 27 21 0 15 Dec 2021
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao Song Licheng Zhang Ruizhe Zhang 32 64 0 14 Dec 2021
Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions Martin Hutzenthaler Arnulf Jentzen Katharina Pohl Adrian Riekert Luca Scarpa MLT 34 6 0 13 Dec 2021
A New Measure of Model Redundancy for Compressed Convolutional Neural Networks Feiqing Huang Yuefeng Si Yao Zheng Guodong Li 39 1 0 09 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 49 16 0 05 Dec 2021
Fast Graph Neural Tangent Kernel via Kronecker Sketching Shunhua Jiang Yunze Man Zhao Song Zheng Yu Danyang Zhuo 29 6 0 04 Dec 2021
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models Tri Dao Beidi Chen Kaizhao Liang Jiaming Yang Zhao Song Atri Rudra Christopher Ré 33 75 0 30 Nov 2021
Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization Thanh Nguyen-Tang Sunil R. Gupta A. Nguyen Svetha Venkatesh OffRL 29 29 0 27 Nov 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 25 18 0 11 Nov 2021
Subquadratic Overparameterization for Shallow Neural Networks Chaehwan Song Ali Ramezani-Kebrya Thomas Pethick Armin Eftekhari V. Cevher 30 31 0 02 Nov 2021
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya Zhang 27 16 0 23 Oct 2021
Quantifying Epistemic Uncertainty in Deep Learning Ziyi Huang H. Lam Haofeng Zhang UQCV BDL UD PER 24 12 0 23 Oct 2021
S-Cyc: A Learning Rate Schedule for Iterative Pruning of ReLU-based Networks Shiyu Liu Chong Min John Tan Mehul Motani CLL 29 4 0 17 Oct 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 36 6 0 15 Oct 2021
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 31 13 0 12 Oct 2021
A global convergence theory for deep ReLU implicit networks via over-parameterization Tianxiang Gao Hailiang Liu Jia Liu Hridesh Rajan Hongyang Gao MLT 31 16 0 11 Oct 2021
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations Jiayao Zhang Hua Wang Weijie J. Su 35 7 0 11 Oct 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao Song Shuo Yang Ruizhe Zhang 38 49 0 09 Oct 2021
EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits Yikun Ban Yuchen Yan A. Banerjee Jingrui He OffRL 37 39 0 07 Oct 2021
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 37 22 0 07 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 43 11 0 06 Oct 2021
Improved architectures and training algorithms for deep operator networks Sizhuang He Hanwen Wang P. Perdikaris AI4CE 52 105 0 04 Oct 2021
Theory of overparametrization in quantum neural networks Martín Larocca Nathan Ju Diego García-Martín Patrick J. Coles M. Cerezo 47 188 0 23 Sep 2021
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks Zhichao Wang Yizhe Zhu 35 18 0 20 Sep 2021
Improving the Robustness of Adversarial Attacks Using an Affine-Invariant Gradient Estimator Wenzhao Xiang Hang Su Chang-rui Liu Yandong Guo Shibao Zheng AAML 29 5 0 13 Sep 2021
Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility Liyun Tu Austin Talbot Neil Gallagher David Carlson DRL 32 2 0 09 Sep 2021
TrAISformer -- A Transformer Network with Sparse Augmented Data Representation and Cross Entropy Loss for AIS-based Vessel Trajectory Prediction Duong Nguyen Ronan Fablet 21 24 0 08 Sep 2021
Impact of GPU uncertainty on the training of predictive deep neural networks Maciej Pietrowski A. Gajda Takuto Yamamoto Taisuke Kobayashi Lana Sinapayen Eiji Watanabe BDL 19 0 0 03 Sep 2021