v1v2v3 (latest)

Characterizing Implicit Bias in Terms of Optimization Geometry

22 February 2018

Papers citing "Characterizing Implicit Bias in Terms of Optimization Geometry"

50 / 290 papers shown

Title
Linear Convergence of Generalized Mirror Descent with Time-Dependent Mirrors Adityanarayanan Radhakrishnan M. Belkin Caroline Uhler 54 9 0 18 Sep 2020
Distributional Generalization: A New Kind of Generalization Preetum Nakkiran Yamini Bansal OOD 85 42 0 17 Sep 2020
Obtaining Adjustable Regularization for Free via Iterate Averaging Jingfeng Wu Vladimir Braverman Lin F. Yang 63 2 0 15 Aug 2020
Why to "grow" and "harvest" deep learning models? I. Kulikovskikh Tarzan Legović VLM 24 0 0 08 Aug 2020
A deep network construction that adapts to intrinsic dimensionality beyond the domain A. Cloninger T. Klock AI4CE 104 14 0 06 Aug 2020
Implicit Regularization via Neural Feature Alignment A. Baratin Thomas George César Laurent R. Devon Hjelm Guillaume Lajoie Pascal Vincent Simon Lacoste-Julien 73 6 0 03 Aug 2020
Understanding Implicit Regularization in Over-Parameterized Single Index Model Jianqing Fan Zhuoran Yang Mengxin Yu 81 18 0 16 Jul 2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth Jason D. Lee Nathan Srebro Daniel Soudry 89 86 0 13 Jul 2020
Statistical and Algorithmic Insights for Semi-supervised Learning with Self-training Samet Oymak Talha Cihad Gulcu 80 20 0 19 Jun 2020
Iterative regularization for convex regularizers C. Molinari Mathurin Massias Lorenzo Rosasco S. Villa 124 0 0 17 Jun 2020
Neural Anisotropy Directions Guillermo Ortiz-Jiménez Apostolos Modas Seyed-Mohsen Moosavi-Dezfooli P. Frossard 105 16 0 17 Jun 2020
Directional Pruning of Deep Neural Networks Shih-Kang Chao Zhanyu Wang Yue Xing Guang Cheng ODL 76 33 0 16 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei Jason D. Lee Tengyu Ma 213 95 0 15 Jun 2020
The role of optimization geometry in single neuron learning Nicholas M. Boffi Stephen Tu Jean-Jacques E. Slotine 54 1 0 15 Jun 2020
To Each Optimizer a Norm, To Each Norm its Generalization Sharan Vaswani Reza Babanezhad Jose Gallego Aaron Mishkin Simon Lacoste-Julien Nicolas Le Roux 66 8 0 11 Jun 2020
Directional convergence and alignment in deep learning Ziwei Ji Matus Telgarsky 70 171 0 11 Jun 2020
Halting Time is Predictable for Large Models: A Universality Property and Average-case Analysis Courtney Paquette B. V. Merrienboer Elliot Paquette Fabian Pedregosa 99 27 0 08 Jun 2020
Classification vs regression in overparameterized regimes: Does the loss function matter? Vidya Muthukumar Adhyyan Narang Vignesh Subramanian M. Belkin Daniel J. Hsu A. Sahai 114 151 0 16 May 2020
Learning the gravitational force law and other analytic functions Atish Agarwala Abhimanyu Das Rina Panigrahy Qiuyi Zhang MLT 43 0 0 15 May 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms Noam Razin Nadav Cohen 81 156 0 13 May 2020
Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime Niladri S. Chatterji Philip M. Long 95 109 0 25 Apr 2020
On Learning Rates and Schrödinger Operators Bin Shi Weijie J. Su Michael I. Jordan 95 61 0 15 Apr 2020
Mirrorless Mirror Descent: A Natural Derivation of Mirror Descent Suriya Gunasekar Blake E. Woodworth Nathan Srebro MDE 111 30 0 02 Apr 2020
Explicit Regularization of Stochastic Gradient Methods through Duality Anant Raj Francis R. Bach 47 6 0 30 Mar 2020
The Implicit Regularization of Stochastic Gradient Flow for Least Squares Alnur Ali Yan Sun Robert Tibshirani 103 77 0 17 Mar 2020
On Alignment in Deep Linear Neural Networks Adityanarayanan Radhakrishnan Eshaan Nichani D. Bernstein Caroline Uhler 47 2 0 13 Mar 2020
Can Implicit Bias Explain Generalization? Stochastic Convex Optimization as a Case Study Assaf Dauber M. Feder Tomer Koren Roi Livni 102 24 0 13 Mar 2020
The Implicit and Explicit Regularization Effects of Dropout Colin Wei Sham Kakade Tengyu Ma 123 118 0 28 Feb 2020
Disentangling Adaptive Gradient Methods from Learning Rates Naman Agarwal Rohan Anil Elad Hazan Tomer Koren Cyril Zhang 109 38 0 26 Feb 2020
A Sample Complexity Separation between Non-Convex and Convex Meta-Learning Nikunj Saunshi Yi Zhang M. Khodak Sanjeev Arora 56 27 0 25 Feb 2020
Hold me tight! Influence of discriminative features on deep network boundaries Guillermo Ortiz-Jiménez Apostolos Modas Seyed-Mohsen Moosavi-Dezfooli P. Frossard AAML 47 50 0 15 Feb 2020
Unique Properties of Flat Minima in Deep Networks Rotem Mulayoff T. Michaeli ODL 59 4 0 11 Feb 2020
Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss Lénaïc Chizat Francis R. Bach MLT 246 341 0 11 Feb 2020
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks Zixiang Chen Yuan Cao Quanquan Gu Tong Zhang MLT 82 10 0 10 Feb 2020
$A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-$\ell_1$-Norm Interpolated Classifiers$ A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum- $\ell_1$ -Norm Interpolated Classifiers Tengyuan Liang Pragya Sur 133 70 0 05 Feb 2020
The Statistical Complexity of Early-Stopped Mirror Descent Tomas Vaskevicius Varun Kanade Patrick Rebeschini 86 23 0 01 Feb 2020
Choosing the Sample with Lowest Loss makes SGD Robust Vatsal Shah Xiaoxia Wu Sujay Sanghavi 65 44 0 10 Jan 2020
Implicit Regularization and Momentum Algorithms in Nonlinearly Parameterized Adaptive Control and Prediction Nicholas M. Boffi Jean-Jacques E. Slotine 83 41 0 31 Dec 2019
On the Bias-Variance Tradeoff: Textbooks Need an Update Brady Neal 43 18 0 17 Dec 2019
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 128 220 0 03 Dec 2019
How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks? Zixiang Chen Yuan Cao Difan Zou Quanquan Gu 77 123 0 27 Nov 2019
Implicit Regularization and Convergence for Weight Normalization Xiaoxia Wu Yan Sun Zhaolin Ren Shanshan Wu Zhiyuan Li Suriya Gunasekar Rachel A. Ward Qiang Liu 155 21 0 18 Nov 2019
Adaptive versus Standard Descent Methods and Robustness Against Adversarial Examples Marc Khoury AAML 53 1 0 09 Nov 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai Jason D. Lee 69 116 0 03 Oct 2019
Implicit Regularization for Optimal Sparse Recovery Tomas Vaskevicius Varun Kanade Patrick Rebeschini 59 103 0 11 Sep 2019
Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective Guan-Horng Liu Evangelos A. Theodorou AI4CE 121 72 0 28 Aug 2019
Bias of Homotopic Gradient Descent for the Hinge Loss Denali Molitor Deanna Needell Rachel A. Ward 39 6 0 26 Jul 2019
Gradient Dynamics of Shallow Univariate ReLU Networks Francis Williams Matthew Trager Claudio Silva Daniele Panozzo Denis Zorin Joan Bruna 70 80 0 18 Jun 2019
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias Stéphane dÁscoli Levent Sagun Joan Bruna Giulio Biroli 95 37 0 16 Jun 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 114 336 0 13 Jun 2019