An overview of gradient descent optimization algorithms

15 September 2016

Papers citing "An overview of gradient descent optimization algorithms"

50 / 1,007 papers shown

Title
Memory Augmented Optimizers for Deep Learning Paul-Aymeric McRae Prasanna Parthasarathi Mahmoud Assran Sarath Chandar ODL 30 3 0 20 Jun 2021
HUMAP: Hierarchical Uniform Manifold Approximation and Projection Wilson E. Marcílio-Jr D. M. Eler F. Paulovich Rafael M. Martins 16 10 0 14 Jun 2021
Category Theory in Machine Learning Dan Shiebler Bruno Gavranović Paul W. Wilson 24 31 0 13 Jun 2021
XBNet : An Extremely Boosted Neural Network Tushar Sarkar LMTD 13 22 0 09 Jun 2021
Learning Stochastic Optimal Policies via Gradient Descent Stefano Massaroli Michael Poli Stefano Peluchetti Jinkyoo Park Atsushi Yamashita Hajime Asama 31 9 0 07 Jun 2021
Neural Architecture Search via Bregman Iterations Leon Bungert Tim Roith Daniel Tenbrinck Martin Burger 6 3 0 04 Jun 2021
Adam in Private: Secure and Fast Training of Deep Neural Networks with Adaptive Moment Estimation Nuttapong Attrapadung Koki Hamada Dai Ikarashi Ryo Kikuchi Takahiro Matsuda Ibuki Mishina Hiraku Morita Jacob C. N. Schuldt 22 27 0 04 Jun 2021
3D map creation using crowdsourced GNSS data Terence Lines A. Basiri 13 10 0 31 May 2021
$Neural Network Training Using $\ell_1$-Regularization and Bi-fidelity Data$ Neural Network Training Using $\ell_1$ -Regularization and Bi-fidelity Data Subhayan De Alireza Doostan 29 24 0 27 May 2021
An Explainable Probabilistic Classifier for Categorical Data Inspired to Quantum Physics E. Guidotti Alfio Ferrara 17 3 0 26 May 2021
2nd-order Updates with 1st-order Complexity M. Zimmer 8 0 0 24 May 2021
Compressing Heavy-Tailed Weight Matrices for Non-Vacuous Generalization Bounds John Y. Shin 24 5 0 23 May 2021
Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective Lun Du Xu Chen Fei Gao Qiang Fu Kunqing Xie Shi Han Dongmei Zhang 20 12 0 17 May 2021
Layerwise Optimization by Gradient Decomposition for Continual Learning Shixiang Tang Dapeng Chen Jinguo Zhu Shijie Yu Wanli Ouyang CLL 24 63 0 17 May 2021
Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers Yilmaz Korkmaz S. Dar Mahmut Yurt Muzaffer Özbey Tolga Çukur ViT MedIm 27 190 0 15 May 2021
ROSEFusion: Random Optimization for Online Dense Reconstruction under Fast Camera Motion Jiazhao Zhang Chenyang Zhu Lintao Zheng Kai Xu 27 45 0 12 May 2021
TAG: Task-based Accumulated Gradients for Lifelong learning Pranshu Malviya B. Ravindran Sarath Chandar CLL 41 5 0 11 May 2021
The impact of the additional features on the performance of regression analysis: a case study on regression analysis of music signal V. N. Aditya Rupaj Kumar 11 0 0 11 May 2021
Modulating Regularization Frequency for Efficient Compression-Aware Model Training Dongsoo Lee S. Kwon Byeongwook Kim Jeongin Yun Baeseong Park Yongkweon Jeon 19 0 0 05 May 2021
Implicit Regularization in Deep Tensor Factorization P. Milanesi Hachem Kadri Stéphane Ayache Thierry Artières 54 9 0 04 May 2021
Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX Chengliang Zhang Junzhe Xia Baichen Yang Huancheng Puyang Wei Wang Ruichuan Chen Istemi Ekin Akkus Paarijaat Aditya Feng Yan FedML 53 39 0 04 May 2021
Crack Semantic Segmentation using the U-Net with Full Attention Strategy F. Lin Jiesheng Yang Jian-Hua Shu Raimar J. Scherer SSeg 13 17 0 29 Apr 2021
Dynamical prediction of two meteorological factors using the deep neural network and the long short-term memory $(2)$ Ki-Hong Shin Jae-Won Jung Ki-Ho Chang Dong-In Lee C. You Kyungsik Kim 23 2 0 28 Apr 2021
One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning Chaosheng Dong Xiaojie Jin Weihao Gao Yijia Wang Hongyi Zhang Xiang Wu Jianchao Yang Xiaobing Liu 28 5 0 27 Apr 2021
On the Importance of 3D Surface Information for Remote Sensing Classification Tasks J. Petrich Ryan M Sander E. Bradley Adam Dawood Shawn Hough 20 1 0 26 Apr 2021
Efficient training of physics-informed neural networks via importance sampling M. A. Nabian R. J. Gladstone Hadi Meidani DiffM PINN 71 223 0 26 Apr 2021
Parallel Physics-Informed Neural Networks via Domain Decomposition K. Shukla Ameya Dilip Jagtap George Karniadakis PINN 103 274 0 20 Apr 2021
Quantum Architecture Search via Deep Reinforcement Learning En-Jui Kuo Yao-Lung L. Fang Samuel Yen-Chi Chen AI4CE 21 83 0 15 Apr 2021
Scale Invariant Monte Carlo under Linear Function Approximation with Curvature based step-size Rahul Madhavan Hemant Makwana 18 0 0 15 Apr 2021
Demystifying BERT: Implications for Accelerator Design Suchita Pati Shaizeen Aga Nuwan Jayasena Matthew D. Sinclair LLMAG 40 17 0 14 Apr 2021
$A Caputo fractional derivative-based algorithm for optimization$ A Caputo fractional derivative-based algorithm for optimization Yeonjong Shin Jérome Darbon George Karniadakis 26 7 0 06 Apr 2021
Training Deep Neural Networks via Branch-and-Bound Yuanwei Wu Ziming Zhang Guanghui Wang ODL 25 0 0 05 Apr 2021
A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions Arnulf Jentzen Adrian Riekert MLT 37 13 0 01 Apr 2021
Wave based damage detection in solid structures using artificial neural networks Frank Wuttke Hao Lyu A. Sattari Z. Rizvi 13 1 0 30 Mar 2021
Multiple-hypothesis CTC-based semi-supervised adaptation of end-to-end speech recognition Cong-Thanh Do R. Doddipatla Thomas Hain 11 6 0 29 Mar 2021
Improving Online Forums Summarization via Hierarchical Unified Deep Neural Network Sansiri Tarnpradab Fereshteh Jafariakinabad K. Hua 21 5 0 25 Mar 2021
Resonant Scanning Design and Control for Fast Spatial Sampling Zhanghao Sun Ronald Quan O. Solgaard 11 2 0 24 Mar 2021
On Imitation Learning of Linear Control Policies: Enforcing Stability and Robustness Constraints via LMI Conditions Aaron J. Havens Bin Hu 9 15 0 24 Mar 2021
Deep Learning for fully automatic detection, segmentation, and Gleason Grade estimation of prostate cancer in multiparametric Magnetic Resonance Images Oscar J. Pellicer-Valero José L. Jiménez V. González-Pérez J. C. Ramón-Borja I. García María Barrios Benito P. P. Gómez J. Rubio-Briones M. J. Rupérez J. D. Martín-Guerrero 20 73 0 23 Mar 2021
CLIP: Cheap Lipschitz Training of Neural Networks Leon Bungert René Raab Tim Roith Leo Schwinn Daniel Tenbrinck 32 32 0 23 Mar 2021
Differentiable Agent-Based Simulation for Gradient-Guided Simulation-Based Optimization Philipp Andelfinger 16 13 0 23 Mar 2021
Spatio-Temporal Neural Network for Fitting and Forecasting COVID-19 Yi-Shuai Niu Wentao Ding Junpeng Hu Wenxu Xu S. Canu 22 2 0 22 Mar 2021
Low Dimensional Landscape Hypothesis is True: DNNs can be Trained in Tiny Subspaces Tao Li Lei Tan Qinghua Tao Yipeng Liu Xiaolin Huang 45 10 0 20 Mar 2021
Learning without gradient descent encoded by the dynamics of a neurobiological model V. George V. Morar Weiwei Yang Jonathan Larson B. Tower Shweti Mahajan Arkin Gupta Christopher M. White Gabriel A. Silva 11 1 0 16 Mar 2021
Surface Topography Characterization Using a Simple Optical Device and Artificial Neural Networks Christoph Angermann Markus Haltmeier Christian Laubichler Steinbjörn Jónsson Matthias Schwab Adéla Moravová C. Kiesling M. Kober W. Fimml 25 7 0 15 Mar 2021
Memristive Stochastic Computing for Deep Learning Parameter Optimization Corey Lammie Jason Eshraghian Wei D. Lu M. R. Azghadi BDL 21 21 0 11 Mar 2021
Quantum machine learning with differential privacy William Watkins Samuel Yen-Chi Chen Shinjae Yoo 29 47 0 10 Mar 2021
Continual Developmental Neurosimulation Using Embodied Computational Agents Bradly Alicea Rishabh Chakrabarty Stefan Dvoretskii Akshara Gopi A. Lim Jesse Parent 23 1 0 07 Mar 2021
Machine Biometrics -- Towards Identifying Machines in a Smart City Environment George K. Sidiropoulos G. Papakostas 14 2 0 25 Feb 2021
Convergence rates for gradient descent in the training of overparameterized artificial neural networks with biases Arnulf Jentzen T. Kröger ODL 28 7 0 23 Feb 2021