Optimization for deep learning: theory and algorithms

19 December 2019

Papers citing "Optimization for deep learning: theory and algorithms"

50 / 85 papers shown

Title
Sharp higher order convergence rates for the Adam optimizer Steffen Dereich Arnulf Jentzen Adrian Riekert ODL 61 0 0 28 Apr 2025
PVBF: A Framework for Mitigating Parameter Variation Imbalance in Online Continual Learning Zelin Tao Hao Deng Mingqing Liu Lijun Zhang Shengjie Zhao CLL 70 0 0 25 Feb 2025
A Theoretical Survey on Foundation Models Shi Fu Yuzhu Chen Yingjie Wang Dacheng Tao 28 0 0 15 Oct 2024
Foundations and Frontiers of Graph Learning Theory Yu Huang Min Zhou Menglin Yang Zhen Wang Muhan Zhang Jie Wang Hong Xie Hao Wang Defu Lian Enhong Chen AI4CE GNN 55 2 0 03 Jul 2024
Predicting the Impact of Model Expansion through the Minima Manifold: A Loss Landscape Perspective Pranshu Malviya Jerry Huang Quentin Fournier Sarath Chandar 62 0 0 24 May 2024
Why Transformers Need Adam: A Hessian Perspective Yushun Zhang Congliang Chen Tian Ding Ziniu Li Ruoyu Sun Zhimin Luo 37 43 0 26 Feb 2024
Fixed width treelike neural networks capacity analysis -- generic activations M. Stojnic 34 3 0 08 Feb 2024
Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks Arnulf Jentzen Adrian Riekert 38 4 0 07 Feb 2024
$\emph{Lifted} RDT based capacity analysis of the 1-hidden layer treelike \emph{sign} perceptrons neural networks$ \emph{Lifted} RDT based capacity analysis of the 1-hidden layer treelike \emph{sign} perceptrons neural networks M. Stojnic 24 1 0 13 Dec 2023
Capacity of the treelike sign perceptrons neural networks with one hidden layer -- RDT based upper bounds M. Stojnic 21 4 0 13 Dec 2023
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 20 1 0 24 Oct 2023
Machine learning in physics: a short guide F. A. Rodrigues PINN AI4CE 13 7 0 16 Oct 2023
Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss T. Getu Georges Kaddoum M. Bennis 40 1 0 13 Sep 2023
Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant Xianbiao Qi Jianan Wang Lei Zhang 15 0 0 15 Jun 2023
ChatGPT-Like Large-Scale Foundation Models for Prognostics and Health Management: A Survey and Roadmaps Yanfang Li Huan Wang Muxia Sun LM&MA AI4TS AI4CE 29 46 0 10 May 2023
Cut your Losses with Squentropy Like Hui M. Belkin S. Wright UQCV 18 8 0 08 Feb 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 54 34 0 27 Jan 2023
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization Hanna Tseran Guido Montúfar ODL 30 0 0 17 Jan 2023
NCVX: A General-Purpose Optimization Solver for Constrained Machine and Deep Learning Buyun Liang Tim Mitchell Ju Sun OOD 18 7 0 03 Oct 2022
On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks Yizhou Liu Weijie J. Su Tongyang Li 24 17 0 29 Sep 2022
Deep Convolutional Neural Network and Transfer Learning for Locomotion Intent Prediction Duong Le S. Cheng Robert Gregg Maani Ghaffari 13 4 0 26 Sep 2022
Defense against Backdoor Attacks via Identifying and Purifying Bad Neurons Mingyuan Fan Yang Liu Cen Chen Ximeng Liu Wenzhong Guo AAML 18 4 0 13 Aug 2022
Winning the Lottery Ahead of Time: Efficient Early Network Pruning John Rachwan Daniel Zügner Bertrand Charpentier Simon Geisler Morgane Ayle Stephan Günnemann 25 24 0 21 Jun 2022
Blind Estimation of a Doubly Selective OFDM Channel: A Deep Learning Algorithm and Theory T. Getu N. Golmie D. Griffith 22 2 0 30 May 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
A Local Convergence Theory for the Stochastic Gradient Descent Method in Non-Convex Optimization With Non-isolated Local Minima Tae-Eon Ko Xiantao Li 22 2 0 21 Mar 2022
Stability and Risk Bounds of Iterative Hard Thresholding Xiao-Tong Yuan P. Li 37 12 0 17 Mar 2022
Symbolic Learning to Optimize: Towards Interpretability and Scalability Wenqing Zheng Tianlong Chen Ting-Kuei Hu Zhangyang Wang 45 19 0 13 Mar 2022
On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problems C. Christof Julia Kowalczyk 30 8 0 23 Feb 2022
Stochastic Modeling of Inhomogeneities in the Aortic Wall and Uncertainty Quantification using a Bayesian Encoder-Decoder Surrogate Sascha Ranftl Malte Rolf-Pissarczyk G. Wolkerstorfer Antonio Pepe Jan Egger W. Linden G. Holzapfel 31 9 0 21 Feb 2022
PFGE: Parsimonious Fast Geometric Ensembling of DNNs Hao Guo Jiyong Jin B. Liu FedML 29 1 0 14 Feb 2022
Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning Said Ouala Steven L. Brunton A. Pascual Bertrand Chapron F. Collard L. Gaultier Ronan Fablet PINN AI4TS AI4CE 18 10 0 11 Feb 2022
Adaptive neighborhood Metric learning Kun Song Junwei Han Gong Cheng Jiwen Lu Feiping Nie 17 27 0 20 Jan 2022
Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks Shaun Li AI4CE 43 0 0 03 Jan 2022
NCVX: A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning Buyun Liang Tim Mitchell Ju Sun 17 3 0 27 Nov 2021
Error Bounds for a Matrix-Vector Product Approximation with Deep ReLU Neural Networks T. Getu 27 2 0 25 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 27 14 0 01 Nov 2021
Early Melanoma Diagnosis with Sequential Dermoscopic Images Zhen Yu Jennifer Nguyen Toàn D. Nguyên J. Kelly C. Mclean Paul Bonnington Lei Zhang Victoria Mar Z. Ge 27 41 0 12 Oct 2021
Momentum Centering and Asynchronous Update for Adaptive Gradient Methods Juntang Zhuang Yifan Ding Tommy M. Tang Nicha Dvornek S. Tatikonda James S. Duncan ODL 19 4 0 11 Oct 2021
A deep neural network for multi-species fish detection using multiple acoustic cameras Garcia Fernandez Guglielmo François Martignac M. Nevoux L. Beaulaton Thomas Corpetti 16 1 0 22 Sep 2021
The loss landscape of deep linear neural networks: a second-order analysis E. M. Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 24 9 0 28 Jul 2021
Advancing biological super-resolution microscopy through deep learning: a brief review Tianjie Yang Yaoru Luo Wei Ji Ge Yang SupR 25 17 0 24 Jun 2021
Accelerating variational quantum algorithms with multiple quantum processors Yuxuan Du Yan Qian Dacheng Tao 9 8 0 24 Jun 2021
The dilemma of quantum neural networks Yan Qian Xinbiao Wang Yuxuan Du Xingyao Wu Dacheng Tao 21 30 0 09 Jun 2021
DeepMoM: Robust Deep Learning With Median-of-Means Shih-Ting Huang Johannes Lederer FedML 21 6 0 28 May 2021
A Probabilistic Approach to Neural Network Pruning Xin-Yao Qian Diego Klabjan 29 16 0 20 May 2021
Deep learning for solution and inversion of structural mechanics and vibrations E. Haghighat A. Bekar E. Madenci R. Juanes PINN AI4CE 20 13 0 18 May 2021
On the Distributional Properties of Adaptive Gradients Z. Zhiyi Liu Ziyin 12 4 0 15 May 2021
Facial Emotion Recognition: State of the Art Performance on FER2013 Yousif Khaireddin Z. Chen 3DH 11 148 0 08 May 2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 164 28 0 22 Apr 2021