Optimization for deep learning: theory and algorithms

19 December 2019

Papers citing "Optimization for deep learning: theory and algorithms"

35 / 85 papers shown

Title
MARINA: Faster Non-Convex Distributed Learning with Compression Eduard A. Gorbunov Konstantin Burlachenko Zhize Li Peter Richtárik 39 109 0 15 Feb 2021
Learning by Turning: Neural Architecture Aware Optimisation Yang Liu Jeremy Bernstein M. Meister Yisong Yue ODL 43 26 0 14 Feb 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 130 167 0 29 Jan 2021
Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges Xin Chen Guannan Qu Yujie Tang S. Low Na Li 24 227 0 27 Jan 2021
Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples Christian Fiedler M. Fornasier T. Klock Michael Rauchensteiner OOD 22 12 0 18 Jan 2021
First-Order Methods for Convex Optimization Pavel Dvurechensky Mathias Staudigl Shimrit Shtern ODL 31 25 0 04 Jan 2021
Recent advances in deep learning theory Fengxiang He Dacheng Tao AI4CE 24 50 0 20 Dec 2020
Recent Theoretical Advances in Non-Convex Optimization Marina Danilova Pavel Dvurechensky Alexander Gasnikov Eduard A. Gorbunov Sergey Guminov Dmitry Kamzolov Innokentiy Shibaev 33 76 0 11 Dec 2020
On the linearity of large non-linear models: when and why the tangent kernel is constant Chaoyue Liu Libin Zhu M. Belkin 21 140 0 02 Oct 2020
Escaping Saddle-Points Faster under Interpolation-like Conditions Abhishek Roy Krishnakumar Balasubramanian Saeed Ghadimi P. Mohapatra 14 1 0 28 Sep 2020
Normalization Techniques in Training DNNs: Methodology, Analysis and Application Lei Huang Jie Qin Yi Zhou Fan Zhu Li Liu Ling Shao AI4CE 12 254 0 27 Sep 2020
Adaptive Hierarchical Hyper-gradient Descent Renlong Jie Junbin Gao A. Vasnev Minh-Ngoc Tran 21 5 0 17 Aug 2020
Deep Learning in Protein Structural Modeling and Design Wenhao Gao S. Mahajan Jeremias Sulam Jeffrey J. Gray 29 159 0 16 Jul 2020
From Symmetry to Geometry: Tractable Nonconvex Problems Yuqian Zhang Qing Qu John N. Wright 26 43 0 14 Jul 2020
AdaScale SGD: A User-Friendly Algorithm for Distributed Training Tyler B. Johnson Pulkit Agrawal Haijie Gu Carlos Guestrin ODL 24 37 0 09 Jul 2020
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK Yuanzhi Li Tengyu Ma Hongyang R. Zhang MLT 20 28 0 09 Jul 2020
Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets Haoxiang Wang Ruoyu Sun Bo Li MLT AI4CE 27 14 0 25 Jun 2020
Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies Yu Huang Yue Chen 3DPC 49 83 0 10 Jun 2020
Random Reshuffling: Simple Analysis with Vast Improvements Konstantin Mishchenko Ahmed Khaled Peter Richtárik 37 131 0 10 Jun 2020
Reinforcement Learning for Caching with Space-Time Popularity Dynamics A. Sadeghi G. Giannakis Gang Wang Fatemeh Sheikholeslami 18 1 0 19 May 2020
The Impact of the Mini-batch Size on the Variance of Gradients in Stochastic Gradient Descent Xin-Yao Qian Diego Klabjan ODL 13 35 0 27 Apr 2020
On Learning Rates and Schrödinger Operators Bin Shi Weijie J. Su Michael I. Jordan 6 59 0 15 Apr 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 16 2 0 23 Mar 2020
Introduction to deep learning Lihi Shiloh-Perl Raja Giryes 27 0 0 29 Feb 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 8 18 0 12 Feb 2020
On the distance between two neural networks and the stability of learning Jeremy Bernstein Arash Vahdat Yisong Yue Xuan Li ODL 200 57 0 09 Feb 2020
Better Theory for SGD in the Nonconvex World Ahmed Khaled Peter Richtárik 13 178 0 09 Feb 2020
Generalization Bounds for High-dimensional M-estimation under Sparsity Constraint Xiao-Tong Yuan Ping Li 11 2 0 20 Jan 2020
Memory capacity of neural networks with threshold and ReLU activations Roman Vershynin 31 21 0 20 Jan 2020
The Practicality of Stochastic Optimization in Imaging Inverse Problems Junqi Tang K. Egiazarian Mohammad Golbabaee Mike Davies 27 30 0 22 Oct 2019
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 238 348 0 14 Jun 2018
Global optimality conditions for deep neural networks Chulhee Yun S. Sra Ali Jadbabaie 128 117 0 08 Jul 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,327 0 05 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 183 1,185 0 30 Nov 2014