v1v2 (latest)

Implicit Regularization in Deep Learning

6 September 2017

Papers citing "Implicit Regularization in Deep Learning"

50 / 105 papers shown

Title
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 109 24 0 18 May 2022
Policy Gradient Method For Robust Reinforcement Learning Yue Wang Shaofeng Zou 132 77 0 15 May 2022
AUC Maximization in the Era of Big Data and AI: A Survey Tianbao Yang Yiming Ying 181 193 0 28 Mar 2022
More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalize Alexander Wei Wei Hu Jacob Steinhardt 112 72 0 11 Mar 2022
Rotationally Equivariant Super-Resolution of Velocity Fields in Two-Dimensional Fluids Using Convolutional Neural Networks Y. Yasuda R. Onishi 74 6 0 22 Feb 2022
On the Implicit Bias Towards Minimal Depth of Deep Neural Networks Tomer Galanti Liane Galanti Ido Ben-Shaul 52 14 0 18 Feb 2022
A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Moments Randall Balestriero Ishan Misra Yann LeCun 76 20 0 16 Feb 2022
Controlling the Complexity and Lipschitz Constant improves polynomial nets Zhenyu Zhu Fabian Latorre Grigorios G. Chrysos Volkan Cevher 57 10 0 10 Feb 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 132 29 0 27 Jan 2022
Leveraging Unlabeled Data to Predict Out-of-Distribution Performance Saurabh Garg Sivaraman Balakrishnan Zachary Chase Lipton Behnam Neyshabur Hanie Sedghi OODD OOD 97 131 0 11 Jan 2022
Automated Reinforcement Learning (AutoRL): A Survey and Open Problems Jack Parker-Holder Raghunandan Rajan Xingyou Song André Biedenkapp Yingjie Miao ... Vu-Linh Nguyen Roberto Calandra Aleksandra Faust Frank Hutter Marius Lindauer AI4CE 112 107 0 11 Jan 2022
Benign Overfitting in Adversarially Robust Linear Classification Jinghui Chen Yuan Cao Quanquan Gu AAML SILM 78 11 0 31 Dec 2021
Global convergence of ResNets: From finite to infinite width using linear parameterization Raphael Barboni Gabriel Peyré Franccois-Xavier Vialard 66 12 0 10 Dec 2021
Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective Margalit Glasgow Honglin Yuan Tengyu Ma FedML 79 45 0 05 Nov 2021
Predictive Model Degrees of Freedom in Linear Regression Bo Luan Yoonkyung Lee Yunzhang Zhu 35 3 0 29 Jun 2021
Stateful ODE-Nets using Basis Function Expansions A. Queiruga N. Benjamin Erichson Liam Hodgkinson Michael W. Mahoney 61 16 0 21 Jun 2021
From inexact optimization to learning via gradient concentration Bernhard Stankewitz Nicole Mücke Lorenzo Rosasco 88 5 0 09 Jun 2021
Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space Sho Sonoda Isao Ishikawa Masahiro Ikeda BDL 51 9 0 09 Jun 2021
The Dynamics of Gradient Descent for Overparametrized Neural Networks Siddhartha Satpathi R. Srikant MLT AI4CE 60 14 0 13 May 2021
Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation Yue Wang Shaofeng Zou Yi Zhou 61 11 0 07 Apr 2021
Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networks Yuqing Li Yaoyu Zhang Chao Ma CML 118 1 0 30 Mar 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 53 79 0 09 Feb 2021
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning Yiding Jiang Pierre Foret Scott Yak Daniel M. Roy H. Mobahi Gintare Karolina Dziugaite Samy Bengio Suriya Gunasekar Isabelle M Guyon Behnam Neyshabur Google Research OOD 74 55 0 14 Dec 2020
On Computability, Learnability and Extractability of Finite State Machines from Recurrent Neural Networks Reda Marzouk 38 2 0 10 Sep 2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth Jason D. Lee Nathan Srebro Daniel Soudry 89 86 0 13 Jul 2020
The Global Landscape of Neural Networks: An Overview Ruoyu Sun Dawei Li Shiyu Liang Tian Ding R. Srikant 84 88 0 02 Jul 2020
Extrapolation for Large-batch Training in Deep Learning Tao R. Lin Lingjing Kong Sebastian U. Stich Martin Jaggi 101 36 0 10 Jun 2020
An Empirical Analysis of the Impact of Data Augmentation on Knowledge Distillation Deepan Das Haley Massa Abhimanyu Kulkarni Theodoros Rekatsinas 60 18 0 06 Jun 2020
Statistical Guarantees for Regularized Neural Networks Mahsa Taheri Fang Xie Johannes Lederer 113 39 0 30 May 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms Noam Razin Nadav Cohen 81 156 0 13 May 2020
On the Benefits of Invariance in Neural Networks Clare Lyle Mark van der Wilk Marta Z. Kwiatkowska Y. Gal Benjamin Bloem-Reddy OOD BDL 84 96 0 01 May 2020
Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime Niladri S. Chatterji Philip M. Long 95 109 0 25 Apr 2020
Robust and On-the-fly Dataset Denoising for Image Classification Jiaming Song Lunjia Hu Michael Auli Yann N. Dauphin Tengyu Ma NoLa OOD 90 13 0 24 Mar 2020
Can Implicit Bias Explain Generalization? Stochastic Convex Optimization as a Case Study Assaf Dauber M. Feder Tomer Koren Roi Livni 102 24 0 13 Mar 2020
Implicit Geometric Regularization for Learning Shapes Amos Gropp Lior Yariv Niv Haim Matan Atzmon Y. Lipman AI4CE 139 863 0 24 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 88 164 0 21 Feb 2020
Sideways: Depth-Parallel Training of Video Models Mateusz Malinowski G. Swirszcz João Carreira Viorica Patraucean MDE 94 14 0 17 Jan 2020
On the Bias-Variance Tradeoff: Textbooks Need an Update Brady Neal 43 18 0 17 Dec 2019
Exact expressions for double descent and implicit regularization via surrogate random design Michal Derezinski Feynman T. Liang Michael W. Mahoney 82 78 0 10 Dec 2019
Observational Overfitting in Reinforcement Learning Xingyou Song Yiding Jiang Stephen Tu Yilun Du Behnam Neyshabur OffRL 129 140 0 06 Dec 2019
How Implicit Regularization of ReLU Neural Networks Characterizes the Learned Function -- Part I: the 1-D Case of Two Layers with Random First Layer Jakob Heiss Josef Teichmann Hanna Wutte MLT 19 5 0 07 Nov 2019
Implicit competitive regularization in GANs Florian Schäfer Hongkai Zheng Anima Anandkumar GAN 82 35 0 13 Oct 2019
Understanding Adversarial Robustness Through Loss Landscape Geometries Vinay Uday Prabhu Dian Ang Yap Joyce Xu John Whaley AAML 58 17 0 22 Jul 2019
Bad Global Minima Exist and SGD Can Reach Them Shengchao Liu Dimitris Papailiopoulos D. Achlioptas 91 81 0 06 Jun 2019
Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks Gauthier Gidel Francis R. Bach Simon Lacoste-Julien AI4CE 84 155 0 30 Apr 2019
Deep Learning for Inverse Problems: Bounds and Regularizers Jaweria Amjad Zhaoyang Lyu M. Rodrigues 38 4 0 31 Jan 2019
On the potential for open-endedness in neural networks N. Guttenberg N. Virgo A. Penn 64 10 0 12 Dec 2018
Analytic Network Learning Kar-Ann Toh 44 9 0 20 Nov 2018
A Modern Take on the Bias-Variance Tradeoff in Neural Networks Brady Neal Sarthak Mittal A. Baratin Vinayak Tantia Matthew Scicluna Simon Lacoste-Julien Ioannis Mitliagkas 87 168 0 19 Oct 2018
A Priori Estimates of the Population Risk for Two-layer Neural Networks Weinan E Chao Ma Lei Wu 92 132 0 15 Oct 2018