Optimization Methods for Large-Scale Machine Learning

15 June 2016

Papers citing "Optimization Methods for Large-Scale Machine Learning"

50 / 1,407 papers shown

Title
Non-Asymptotic Analysis of Online Multiplicative Stochastic Gradient Descent Riddhiman Bhattacharya Tiefeng Jiang 21 0 0 14 Dec 2021
Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions Martin Hutzenthaler Arnulf Jentzen Katharina Pohl Adrian Riekert Luca Scarpa MLT 34 6 0 13 Dec 2021
A Novel Sequential Coreset Method for Gradient Descent Algorithms Jiawei Huang Ru Huang Wenjie Liu N. Freris Huihua Ding 29 16 0 05 Dec 2021
Regularized Newton Method with Global $O(1/k^2)$ Convergence Konstantin Mishchenko 17 39 0 03 Dec 2021
On Large Batch Training and Sharp Minima: A Fokker-Planck Perspective Xiaowu Dai Yuhua Zhu 27 4 0 02 Dec 2021
Improving Differentially Private SGD via Randomly Sparsified Gradients Junyi Zhu Matthew B. Blaschko 30 5 0 01 Dec 2021
Adaptive First- and Second-Order Algorithms for Large-Scale Machine Learning Sanae Lotfi Tiphaine Bonniot de Ruisselet D. Orban Andrea Lodi ODL 27 1 0 29 Nov 2021
An Optimization Framework for Federated Edge Learning Yangchen Li Ying Cui Vincent K. N. Lau FedML 19 7 0 26 Nov 2021
Random-reshuffled SARAH does not need a full gradient computations Aleksandr Beznosikov Martin Takáč 31 7 0 26 Nov 2021
BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule Miao Zhang Jilin Hu Steven W. Su Shirui Pan Xiaojun Chang B. Yang Gholamreza Haffari OOD 45 15 0 25 Nov 2021
MIO : Mutual Information Optimization using Self-Supervised Binary Contrastive Learning Siladittya Manna Umapada Pal Saumik Bhattacharya SSL 35 1 0 24 Nov 2021
Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning Zhenhuan Yang Yunwen Lei Puyu Wang Tianbao Yang Yiming Ying 16 26 0 23 Nov 2021
Variance Reduction in Deep Learning: More Momentum is All You Need Lionel Tondji S. Kashubin Moustapha Cissé ODL 21 1 0 23 Nov 2021
Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits Hao Chen Lili Zheng Raed Al Kontar Garvesh Raskutti 24 3 0 19 Nov 2021
Online Estimation and Optimization of Utility-Based Shortfall Risk Vishwajit Hegde Arvind S. Menon L. A. Prashanth Krishna Jagannathan 24 2 0 16 Nov 2021
Stochastic Gradient Line Bayesian Optimization for Efficient Noise-Robust Optimization of Parameterized Quantum Circuits Shiro Tamiya H. Yamasaki 26 24 0 15 Nov 2021
Bolstering Stochastic Gradient Descent with Model Building Ş. Birbil Özgür Martin Gönenç Onay Figen Oztoprak ODL 21 1 0 13 Nov 2021
Stationary Behavior of Constant Stepsize SGD Type Algorithms: An Asymptotic Characterization Zaiwei Chen Shancong Mou S. T. Maguluri 17 13 0 11 Nov 2021
Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum J. Kim Panos Toulis Anastasios Kyrillidis 28 8 0 11 Nov 2021
Learning Rates for Nonconvex Pairwise Learning Shaojie Li Yong Liu 32 2 0 09 Nov 2021
AGGLIO: Global Optimization for Locally Convex Functions Debojyoti Dey B. Mukhoty Purushottam Kar 16 2 0 06 Nov 2021
Finding the Optimal Dynamic Treatment Regime Using Smooth Fisher Consistent Surrogate Loss Nilanjana Laha Aaron Sonabend-W Rajarshi Mukherjee Tianxi Cai 14 1 0 03 Nov 2021
An Asymptotic Analysis of Minibatch-Based Momentum Methods for Linear Regression Models Yuan Gao Xuening Zhu Haobo Qi Guodong Li Riquan Zhang Hansheng Wang 23 3 0 02 Nov 2021
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 32 14 0 01 Nov 2021
Multi-Task Learning based Convolutional Models with Curriculum Learning for the Anisotropic Reynolds Stress Tensor in Turbulent Duct Flow Haitz Sáez de Ocáriz Borde David Sondak P. Protopapas AI4CE 22 3 0 30 Oct 2021
Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima Guangyuan Shi Jiaxin Chen Wenlong Zhang Li-Ming Zhan Xiao-Ming Wu CLL 36 152 0 30 Oct 2021
Dynamic Differential-Privacy Preserving SGD Jian Du Song Li Xiangyi Chen Siheng Chen Mingyi Hong 27 31 0 30 Oct 2021
Efficient Meta Subspace Optimization Yoni Choukroun Michael Katz 25 1 0 28 Oct 2021
How Important is Importance Sampling for Deep Budgeted Training? Eric Arazo Diego Ortego Paul Albert Noel E. O'Connor Kevin McGuinness 23 7 0 27 Oct 2021
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums Rui Pan Haishan Ye Tong Zhang 22 14 0 27 Oct 2021
Accelerated Almost-Sure Convergence Rates for Nonconvex Stochastic Gradient Descent using Stochastic Learning Rates Theodoros Mamalis D. Stipanović R. Tao 26 2 0 25 Oct 2021
Boosting Federated Learning in Resource-Constrained Networks Mohamed Yassine Boukhari Akash Dhasade Anne-Marie Kermarrec Rafael Pires Othmane Safsafi Rishi Sharma FedML 12 0 0 21 Oct 2021
Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent Sharan Vaswani Benjamin Dubois-Taine Reza Babanezhad 53 11 0 21 Oct 2021
Utilizing Redundancy in Cost Functions for Resilience in Distributed Optimization and Learning Shuo Liu Nirupam Gupta Nitin H. Vaidya 32 0 0 21 Oct 2021
A Data-Centric Optimization Framework for Machine Learning Oliver Rausch Tal Ben-Nun Nikoli Dryden Andrei Ivanov Shigang Li Torsten Hoefler AI4CE 22 16 0 20 Oct 2021
Stochastic Learning Rate Optimization in the Stochastic Approximation and Online Learning Settings Theodoros Mamalis D. Stipanović P. Voulgaris 17 4 0 20 Oct 2021
Optimal randomized classification trees R. Blanquero E. Carrizosa Antonios Tsourdos Dolores Romero Morales 19 47 0 19 Oct 2021
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization Tao Sun Huaming Ling Zuoqiang Shi Dongsheng Li Bao Wang ODL 27 13 0 18 Oct 2021
A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization C. Alecsa 30 0 0 16 Oct 2021
Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining Tim Kaler Nickolas Stathas Anne Ouyang A. Iliopoulos Tao B. Schardl C. E. Leiserson Jie Chen GNN 70 53 0 16 Oct 2021
Resource-constrained Federated Edge Learning with Heterogeneous Data: Formulation and Analysis Yi Liu Yuanshao Zhu James J. Q. Yu FedML 32 28 0 14 Oct 2021
Adaptive Differentially Private Empirical Risk Minimization Xiaoxia Wu Lingxiao Wang Irina Cristali Quanquan Gu Rebecca Willett 40 6 0 14 Oct 2021
Adaptive Elastic Training for Sparse Deep Learning on Heterogeneous Multi-GPU Servers Yujing Ma Florin Rusu Kesheng Wu A. Sim 46 3 0 13 Oct 2021
Gradual Federated Learning with Simulated Annealing L. Nguyen Junhan Kim B. Shim FedML 21 10 0 11 Oct 2021
Convergence of Random Reshuffling Under The Kurdyka-Łojasiewicz Inequality Xiao Li Andre Milzarek Junwen Qiu 26 20 0 10 Oct 2021
Combining Differential Privacy and Byzantine Resilience in Distributed SGD R. Guerraoui Nirupam Gupta Rafael Pinot Sébastien Rouault John Stephan FedML 43 4 0 08 Oct 2021
Does Momentum Change the Implicit Regularization on Separable Data? Bohan Wang Qi Meng Huishuai Zhang Ruoyu Sun Wei Chen Zhirui Ma Tie-Yan Liu 47 15 0 08 Oct 2021
Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect Yuqing Wang Minshuo Chen T. Zhao Molei Tao AI4CE 64 40 0 07 Oct 2021
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 39 22 0 07 Oct 2021
Global Convergence and Stability of Stochastic Gradient Descent V. Patel Shushu Zhang Bowen Tian 33 22 0 04 Oct 2021