Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition

6 March 2015

Furong Huang

Papers citing "Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition"

50 / 211 papers shown

Title
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 28 8 0 18 Feb 2022
Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions Aaron Mishkin Arda Sahiner Mert Pilanci OffRL 77 30 0 02 Feb 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 25 34 0 20 Jan 2022
Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems C. J. Li Michael I. Jordan 21 2 0 29 Dec 2021
Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions Martin Hutzenthaler Arnulf Jentzen Katharina Pohl Adrian Riekert Luca Scarpa MLT 34 6 0 13 Dec 2021
Escape saddle points by a simple gradient-descent based algorithm Chenyi Zhang Tongyang Li ODL 31 15 0 28 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 30 14 0 01 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 43 1 0 25 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 41 11 0 06 Oct 2021
On the Estimation Bias in Double Q-Learning Zhizhou Ren Guangxiang Zhu Haotian Hu Beining Han Jian-Hai Chen Chongjie Zhang 24 17 0 29 Sep 2021
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 89 72 0 29 Sep 2021
A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions Arnulf Jentzen Adrian Riekert 33 13 0 10 Aug 2021
Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization Yuetian Luo Xudong Li Anru R. Zhang 33 9 0 03 Aug 2021
The loss landscape of deep linear neural networks: a second-order analysis E. M. Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 24 9 0 28 Jul 2021
Physics-constrained Deep Learning for Robust Inverse ECG Modeling Jianxin Xie B. Yao 30 21 0 26 Jul 2021
Activated Gradients for Deep Neural Networks Mei Liu Liangming Chen Xiaohao Du Long Jin Mingsheng Shang ODL AI4CE 35 135 0 09 Jul 2021
Distributed stochastic optimization with large delays Zhengyuan Zhou P. Mertikopoulos Nicholas Bambos Peter Glynn Yinyu Ye 25 9 0 06 Jul 2021
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 42 75 0 28 Jun 2021
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity Shiwei Liu Tianlong Chen Zahra Atashgahi Xiaohan Chen Ghada Sokar Elena Mocanu Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu OOD 31 49 0 28 Jun 2021
Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization Tian-Chun Ye S. Du 21 46 0 27 Jun 2021
Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent Spencer Frei Quanquan Gu 26 25 0 25 Jun 2021
Gradient play in stochastic games: stationary points, convergence, and sample complexity Runyu Zhang Zhaolin Ren Na Li 28 43 0 01 Jun 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 22 4 0 21 May 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 30 194 0 06 May 2021
Sharp Global Guarantees for Nonconvex Low-rank Recovery in the Noisy Overparameterized Regime Richard Y. Zhang 47 25 0 21 Apr 2021
Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis Zezhong Zhang Guangxu Zhu Rui-cang Wang Vincent K. N. Lau Kaibin Huang 33 31 0 20 Apr 2021
An Efficient Algorithm for Deep Stochastic Contextual Bandits Tan Zhu Guannan Liang Chunjiang Zhu HaiNing Li J. Bi 37 1 0 12 Apr 2021
A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions Arnulf Jentzen Adrian Riekert MLT 34 13 0 01 Apr 2021
Significance tests of feature relevance for a black-box learner Ben Dai Xiaotong Shen Wei Pan 24 25 0 02 Mar 2021
Stochastic Gradient Langevin Dynamics with Variance Reduction Zhishen Huang Stephen Becker 15 7 0 12 Feb 2021
Stability of SGD: Tightness Analysis and Improved Bounds Yikai Zhang Wenjia Zhang Sammy Bald Vamsi Pingali Chao Chen Mayank Goswami MLT 27 36 0 10 Feb 2021
Learning from Sparse Demonstrations Wanxin Jin Todd D. Murphey Dana Kulić Neta Ezer Shaoshuai Mou 19 35 0 05 Aug 2020
Regularized linear autoencoders recover the principal components, eventually Xuchan Bao James Lucas Sushant Sachdeva Roger C. Grosse 42 29 0 13 Jul 2020
Hyperparameter Ensembles for Robustness and Uncertainty Quantification F. Wenzel Jasper Snoek Dustin Tran Rodolphe Jenatton UQCV 33 204 0 24 Jun 2020
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Ayush Sekhari Karthik Sridharan 87 53 0 24 Jun 2020
A Convergent and Dimension-Independent Min-Max Optimization Algorithm Vijay Keswani Oren Mangoubi Sushant Sachdeva Nisheeth K. Vishnoi 15 1 0 22 Jun 2020
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems P. Mertikopoulos Nadav Hallak Ali Kavis V. Cevher 30 85 0 19 Jun 2020
Self-training Avoids Using Spurious Features Under Domain Shift Yining Chen Colin Wei Ananya Kumar Tengyu Ma OOD 29 84 0 17 Jun 2020
The limits of min-max optimization algorithms: convergence to spurious non-critical sets Ya-Ping Hsieh P. Mertikopoulos V. Cevher 35 81 0 16 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei J. Lee Tengyu Ma 29 93 0 15 Jun 2020
An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias Lu Yu Krishnakumar Balasubramanian S. Volgushev Murat A. Erdogdu 35 50 0 14 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 14 37 0 12 Jun 2020
Directional convergence and alignment in deep learning Ziwei Ji Matus Telgarsky 20 163 0 11 Jun 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 39 273 0 01 Jun 2020
Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent Tian Tong Cong Ma Yuejie Chi 27 115 0 18 May 2020
High-Dimensional Robust Mean Estimation via Gradient Descent Yu Cheng Ilias Diakonikolas Rong Ge Mahdi Soltanolkotabi 17 31 0 04 May 2020
Likelihood landscape and maximum likelihood estimation for the discrete orbit recovery model Z. Fan Yi Sun Tianhao Wang Yihong Wu 30 18 0 31 Mar 2020
The Landscape of Matrix Factorization Revisited Hossein Valavi Sulin Liu Peter J. Ramadge 17 5 0 27 Feb 2020
Implicit Geometric Regularization for Learning Shapes Amos Gropp Lior Yariv Niv Haim Matan Atzmon Y. Lipman AI4CE 60 852 0 24 Feb 2020
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning Yeming Wen Dustin Tran Jimmy Ba OOD FedML UQCV 32 482 0 17 Feb 2020