v1v2v3 (latest)

FedNest: Federated Bilevel, Minimax, and Compositional Optimization

4 May 2022

Davoud Ataee Tarzanagh

Mingchen Li

Christos Thrampoulidis

Samet Oymak

FedML

ArXiv (abs)PDF HTML Github (16★)

Papers citing "FedNest: Federated Bilevel, Minimax, and Compositional Optimization"

50 / 66 papers shown

Title
Fully First-Order Methods for Decentralized Bilevel Optimization Xiaoyu Wang Xuxing Chen Shiqian Ma Tong Zhang 104 0 0 25 Oct 2024
Randomized Block-Coordinate Optimistic Gradient Algorithms for Root-Finding Problems Quoc Tran-Dinh Yang Luo 186 8 0 08 Jan 2023
AutoBalance: Optimized Loss Functions for Imbalanced Data Mingchen Li Xuechen Zhang Christos Thrampoulidis Jiasi Chen Samet Oymak 56 68 0 04 Jan 2022
Federated Learning Based on Dynamic Regularization D. A. E. Acar Yue Zhao Ramon Matas Navarro Matthew Mattina P. Whatmough Venkatesh Saligrama FedML 81 781 0 08 Nov 2021
FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation Yan Shen Jianguo Du Hao Zhang Benyu Zhang Zhanghexuan Ji Mingchen Gao FedML 73 13 0 16 Oct 2021
Compositional federated learning: Applications in distributionally robust averaging and meta learning Feihu Huang Junyi Li FedML 71 15 0 21 Jun 2021
A Decentralized Adaptive Momentum Method for Solving a Class of Min-Max Optimization Problems Babak Barazandeh Tianjian Huang George Michailidis 65 12 0 10 Jun 2021
Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing M. Khodak Renbo Tu Tian Li Liam Li Maria-Florina Balcan Virginia Smith Ameet Talwalkar FedML 98 78 0 08 Jun 2021
Solving a class of non-convex min-max games using adaptive momentum methods Babak Barazandeh Davoud Ataee Tarzanagh George Michailidis 87 13 0 26 Apr 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 127 96 0 02 Mar 2021
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency Yuyang Deng M. Mahdavi 95 61 0 25 Feb 2021
Distributionally Robust Federated Averaging Yuyang Deng Mohammad Mahdi Kamani M. Mahdavi FedML 59 142 0 25 Feb 2021
A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentum Prashant Khanduri Siliang Zeng Mingyi Hong Hoi-To Wai Zhaoran Wang Zhuoran Yang 89 132 0 15 Feb 2021
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients A. Mitra Rayana H. Jaafar George J. Pappas Hamed Hassani FedML 110 160 0 14 Feb 2021
Efficient Algorithms for Federated Saddle Point Optimization Charlie Hou K. K. Thekumparampil Giulia Fanti Sewoong Oh FedML 88 23 0 12 Feb 2021
Lower Bounds and Accelerated Algorithms for Bilevel Optimization Kaiyi Ji Yingbin Liang 64 58 0 07 Feb 2021
Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond Risheng Liu Jiaxin Gao Jin Zhang Deyu Meng Zhouchen Lin AI4CE 143 229 0 27 Jan 2021
Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization Jelena Diakonikolas C. Daskalakis Michael I. Jordan 84 145 0 31 Oct 2020
Bilevel Optimization: Convergence Analysis and Enhanced Design Kaiyi Ji Junjie Yang Yingbin Liang 217 262 0 15 Oct 2020
Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee Junyi Li Bin Gu Heng-Chiao Huang 68 41 0 01 Sep 2020
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization Jianyu Wang Qinghua Liu Hao Liang Gauri Joshi H. Vincent Poor MoMe FedML 73 1,359 0 15 Jul 2020
A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic Mingyi Hong Hoi-To Wai Zhaoran Wang Zhuoran Yang 82 140 0 10 Jul 2020
On the Iteration Complexity of Hypergradient Computation Riccardo Grazzi Luca Franceschi Massimiliano Pontil Saverio Salzo 122 197 0 29 Jun 2020
Robust Federated Learning: The Case of Affine Distribution Shifts Amirhossein Reisizadeh Farzan Farnia Ramtin Pedarsani Ali Jadbabaie FedML OOD 95 166 0 16 Jun 2020
FedGAN: Federated Generative Adversarial Networks for Distributed Data M. Rasouli Tao Sun Ram Rajagopal FedML 102 145 0 12 Jun 2020
A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton Risheng Liu Pan Mu Xiaoming Yuan Shangzhi Zeng Jin Zhang 70 131 0 07 Jun 2020
FedSplit: An algorithmic framework for fast federated optimization Reese Pathak Martin J. Wainwright FedML 219 184 0 11 May 2020
A Finite Time Analysis of Two Time-Scale Actor Critic Methods Yue Wu Weitong Zhang Pan Xu Quanquan Gu 179 149 0 04 May 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 190 1,452 0 29 Feb 2020
Provable Representation Learning for Imitation Learning via Bi-level Optimization Sanjeev Arora S. Du Sham Kakade Yuping Luo Nikunj Saunshi 82 61 0 24 Feb 2020
Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets Mingrui Liu Youssef Mroueh Jerret Ross Wei Zhang Xiaodong Cui Payel Das Tianbao Yang ODL 83 63 0 26 Dec 2019
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 285 6,307 0 10 Dec 2019
Optimizing Millions of Hyperparameters by Implicit Differentiation Jonathan Lorraine Paul Vicol David Duvenaud DD 139 416 0 06 Nov 2019
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification T. Hsu Qi Matthew Brown FedML 150 1,166 0 13 Sep 2019
First Analysis of Local GD on Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik FedML 87 172 0 10 Sep 2019
On the Convergence of FedAvg on Non-IID Data Xiang Li Kaixuan Huang Wenhao Yang Shusen Wang Zhihua Zhang FedML 182 2,356 0 04 Jul 2019
Efficient Algorithms for Smooth Minimax Optimization K. K. Thekumparampil Prateek Jain Praneeth Netrapalli Sewoong Oh 100 191 0 02 Jul 2019
Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations Debraj Basu Deepesh Data C. Karakuş Suhas Diggavi MQ 69 406 0 06 Jun 2019
On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems Tianyi Lin Chi Jin Michael I. Jordan 137 508 0 02 Jun 2019
Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods Maher Nouiehed Maziar Sanjabi Tianjian Huang Jason D. Lee Meisam Razaviyayn 100 344 0 21 Feb 2019
Agnostic Federated Learning M. Mohri Gary Sivek A. Suresh FedML 139 941 0 01 Feb 2019
DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimization Parvin Nazari Davoud Ataee Tarzanagh George Michailidis ODL 89 68 0 25 Jan 2019
A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach Aryan Mokhtari Asuman Ozdaglar S. Pattathil 106 328 0 24 Jan 2019
Federated Optimization in Heterogeneous Networks Tian Li Anit Kumar Sahu Manzil Zaheer Maziar Sanjabi Ameet Talwalkar Virginia Smith FedML 192 5,247 0 14 Dec 2018
Truncated Back-propagation for Bilevel Optimization Amirreza Shaban Ching-An Cheng Nathan Hatch Byron Boots 111 266 0 25 Oct 2018
Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learning Hassan Rafique Mingrui Liu Qihang Lin Tianbao Yang 92 111 0 04 Oct 2018
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms Jianyu Wang Gauri Joshi 192 349 0 22 Aug 2018
Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning Hao Yu Sen Yang Shenghuo Zhu MoMe FedML 90 609 0 17 Jul 2018
The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization C. Daskalakis Ioannis Panageas 83 256 0 11 Jul 2018
DARTS: Differentiable Architecture Search Hanxiao Liu Karen Simonyan Yiming Yang 208 4,375 0 24 Jun 2018