DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimization

25 January 2019

Parvin Nazari

Davoud Ataee Tarzanagh

George Michailidis

ODL

ArXiv PDF HTML

Papers citing "DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimization"

35 / 35 papers shown

Title
Dynamic Regret of Adaptive Gradient Methods for Strongly Convex Problems Parvin Nazari E. Khorram ODL 74 3 0 04 Sep 2022
Adaptive First-and Zeroth-order Methods for Weakly Convex Stochastic Optimization Problems Parvin Nazari Davoud Ataee Tarzanagh George Michailidis ODL 36 13 0 19 May 2020
Dynamic Local Regret for Non-convex Online Forecasting Sergul Aydore Tianhao Zhu Dean Phillips Foster AI4TS 35 17 0 16 Oct 2019
Central Server Free Federated Learning over Single-sided Trust Social Networks Chaoyang He Conghui Tan Hanlin Tang Shuang Qiu Ji Liu FedML 39 75 0 11 Oct 2019
Introduction to Online Convex Optimization Elad Hazan OffRL 163 1,928 0 07 Sep 2019
Online Learning over Dynamic Graphs via Distributed Proximal Gradient Algorithm Rishabh Dixit Amrit Singh Bedi K. Rajawat 28 32 0 16 May 2019
On the Convergence of Adam and Beyond Sashank J. Reddi Satyen Kale Surinder Kumar 87 2,498 0 19 Apr 2019
On the Convergence Proof of AMSGrad and a New Version Phuong T. Tran L. T. Phong ODL 58 87 0 07 Apr 2019
Online Learning with Continuous Variations: Dynamic Regret and Reductions Ching-An Cheng Jonathan Lee Ken Goldberg Byron Boots 53 16 0 19 Feb 2019
Decentralized Online Learning: Take Benefits from Others' Data without Sharing Your Own to Track Global Trend Wendi Wu Zongren Li Yawei Zhao Chenkai Yu P. Zhao Ji Liu FedML 51 16 0 29 Jan 2019
On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization Dongruo Zhou Yiqi Tang Yuan Cao Ziyan Yang Quanquan Gu 52 151 0 16 Aug 2018
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization Xiangyi Chen Sijia Liu Ruoyu Sun Mingyi Hong 53 323 0 08 Aug 2018
Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration Soham De Anirbit Mukherjee Enayat Ullah 49 101 0 18 Jul 2018
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes Rachel A. Ward Xiaoxia Wu Léon Bottou ODL 59 364 0 05 Jun 2018
D $^2$ : Decentralized Training over Decentralized Data Hanlin Tang Xiangru Lian Ming Yan Ce Zhang Ji Liu 31 350 0 19 Mar 2018
Zeroth Order Nonconvex Multi-Agent Optimization over Networks Davood Hajinezhad Mingyi Hong Alfredo García 51 80 0 27 Oct 2017
Decentralized Online Learning with Kernels Alec Koppel Santiago Paternain C. Richard Alejandro Ribeiro 56 51 0 11 Oct 2017
Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization A. Nedić Alexander Olshevsky Michael G. Rabbat 58 509 0 26 Sep 2017
Efficient Regret Minimization in Non-Convex Games Elad Hazan Karan Singh Cyril Zhang 91 96 0 31 Jul 2017
Collaborative Deep Learning in Fixed Topology Networks Zhanhong Jiang Aditya Balu Chinmay Hegde Soumik Sarkar FedML 54 180 0 23 Jun 2017
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang Ji Liu 50 1,227 0 25 May 2017
The Marginal Value of Adaptive Gradient Methods in Machine Learning Ashia Wilson Rebecca Roelofs Mitchell Stern Nathan Srebro Benjamin Recht ODL 56 1,029 0 23 May 2017
An Online Optimization Approach for Multi-Agent Tracking of Dynamic Parameters in the Presence of Adversarial Noise Shahin Shahrampour Ali Jadbabaie 53 24 0 21 Feb 2017
Communication-Efficient Algorithms for Decentralized and Stochastic Optimization Guanghui Lan Soomin Lee Yi Zhou 71 218 0 14 Jan 2017
Distributed Online Optimization in Dynamic Environments Using Mirror Descent Shahin Shahrampour Ali Jadbabaie 216 280 0 09 Sep 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 758 36,781 0 25 Aug 2016
On Nonconvex Decentralized Gradient Descent Jinshan Zeng W. Yin 71 191 0 20 Aug 2016
Improved Dynamic Regret for Non-degenerate Functions Lijun Zhang Tianbao Yang Jinfeng Yi Jing Rong Zhi Zhou 223 127 0 13 Aug 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 390 17,453 0 17 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 193,814 0 10 Dec 2015
Online Distributed Optimization on Dynamic Networks Saghar Hosseini Airlie Chapman M. Mesbahi 67 145 0 22 Dec 2014
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.7K 150,006 0 22 Dec 2014
Non-stationary Stochastic Optimization Omar Besbes Y. Gur A. Zeevi 171 433 0 20 Jul 2013
ADADELTA: An Adaptive Learning Rate Method Matthew D. Zeiler ODL 134 6,626 0 22 Dec 2012
Adaptive Bound Optimization for Online Convex Optimization H. B. McMahan Matthew J. Streeter ODL 96 387 0 26 Feb 2010