Last-iterate Convergence of Decentralized Optimistic Gradient Descent/Ascent in Infinite-horizon Competitive Markov Games

8 February 2021

Papers citing "Last-iterate Convergence of Decentralized Optimistic Gradient Descent/Ascent in Infinite-horizon Competitive Markov Games"

50 / 56 papers shown

Title
Properties of Fixed Points of Generalised Extra Gradient Methods Applied to Min-Max Problems Amir Ali Farzin Yuen-Man Pun Philipp Braun Iman Shames 23 0 0 03 Apr 2025
Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guarantees Yongtao Wu Luca Viano Yihang Chen Zhenyu Zhu Kimon Antonakopoulos Quanquan Gu V. Cevher 54 0 0 18 Feb 2025
Decentralized Online Learning in General-Sum Stackelberg Games Yaolong Yu Haipeng Chen 27 0 0 06 May 2024
Linear Convergence of Independent Natural Policy Gradient in Games with Entropy Regularization Youbang Sun Tao-Wen Liu P. R. Kumar Shahin Shahrampour 37 0 0 04 May 2024
$$\widetilde{O}(T^{-1})$ Convergence to (Coarse) Correlated Equilibria in Full-Information General-Sum Markov Games$ $\widetilde{O}(T^{-1})$ Convergence to (Coarse) Correlated Equilibria in Full-Information General-Sum Markov Games Weichao Mao Haoran Qiu Chen Wang Hubertus Franke Zbigniew T. Kalbarczyk Tamer Basar 30 0 0 02 Feb 2024
Near-Optimal Policy Optimization for Correlated Equilibrium in General-Sum Markov Games Yang Cai Haipeng Luo Chen-Yu Wei Weiqiang Zheng 26 7 0 26 Jan 2024
Optimistic Policy Gradient in Multi-Player Markov Games with a Single Controller: Convergence Beyond the Minty Property Ioannis Anagnostides Ioannis Panageas Gabriele Farina T. Sandholm 33 3 0 19 Dec 2023
Scalable and Independent Learning of Nash Equilibrium Policies in $n$ -Player Stochastic Games with Unknown Independent Chains Tiancheng Qin S. Rasoul Etesami 21 2 0 04 Dec 2023
Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems Juno Kim Kakei Yamamoto Kazusato Oko Zhuoran Yang Taiji Suzuki 34 9 0 02 Dec 2023
Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm Miaoxi Zhu Li Shen Bo Du Dacheng Tao 23 6 0 31 Oct 2023
Provably Fast Convergence of Independent Natural Policy Gradient for Markov Potential Games Youbang Sun Tao-Wen Liu Ruida Zhou P. R. Kumar Shahin Shahrampour 33 11 0 15 Oct 2023
Global Convergence of Policy Gradient Methods in Reinforcement Learning, Games and Control Shicong Cen Yuejie Chi 42 1 0 08 Oct 2023
Local and adaptive mirror descents in extensive-form games Côme Fiegel Pierre Ménard Tadashi Kozuno Rémi Munos Vianney Perchet Michal Valko 19 2 0 01 Sep 2023
Multi-Player Zero-Sum Markov Games with Networked Separable Interactions Chanwoo Park Kaipeng Zhang Asuman Ozdaglar 30 8 0 13 Jul 2023
Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPs Dongsheng Ding Chen-Yu Wei Kaipeng Zhang Alejandro Ribeiro 40 19 0 20 Jun 2023
Zero-sum Polymatrix Markov Games: Equilibrium Collapse and Efficient Computation of Nash Equilibria Fivos Kalogiannis Ioannis Panageas 32 8 0 23 May 2023
Sublinear Convergence Rates of Extragradient-Type Methods: A Survey on Classical and Recent Developments Quoc Tran-Dinh 35 7 0 30 Mar 2023
Uncoupled and Convergent Learning in Two-Player Zero-Sum Markov Games with Bandit Feedback Yang Cai Haipeng Luo Chen-Yu Wei Weiqiang Zheng 26 17 0 05 Mar 2023
A Finite-Sample Analysis of Payoff-Based Independent Learning in Zero-Sum Stochastic Games Zaiwei Chen Kaipeng Zhang Eric Mazumdar Asuman Ozdaglar Adam Wierman 45 6 0 03 Mar 2023
Can We Find Nash Equilibria at a Linear Rate in Markov Games? Zhuoqing Song Jason D. Lee Zhuoran Yang 29 8 0 03 Mar 2023
Population-size-Aware Policy Optimization for Mean-Field Games Pengdeng Li Xinrun Wang Shuxin Li Hau Chan Bo An 21 2 0 07 Feb 2023
Decentralized model-free reinforcement learning in stochastic games with average-reward objective Romain Cravic Nicolas Gast B. Gaujal 29 2 0 13 Jan 2023
Adapting to game trees in zero-sum imperfect information games Côme Fiegel Pierre Ménard Tadashi Kozuno Rémi Munos Vianney Perchet Michal Valko 24 9 0 23 Dec 2022
Asynchronous Gradient Play in Zero-Sum Multi-agent Games Ruicheng Ao Shicong Cen Yuejie Chi 45 5 0 16 Nov 2022
Symmetric (Optimistic) Natural Policy Gradient for Multi-agent Learning with Parameter Convergence S. Pattathil Kaipeng Zhang Asuman Ozdaglar 21 12 0 23 Oct 2022
On the convergence of policy gradient methods to Nash equilibria in general stochastic games Angeliki Giannou Kyriakos Lotidis P. Mertikopoulos Emmanouil-Vasileios Vlatakis-Gkaragkounis 29 17 0 17 Oct 2022
Decentralized Policy Gradient for Nash Equilibria Learning of General-sum Stochastic Games Yan Chen Taoying Li 16 2 0 14 Oct 2022
Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games Shicong Cen Yuejie Chi S. Du Lin Xiao 51 35 0 03 Oct 2022
$$O(T^{-1})$ Convergence of Optimistic-Follow-the-Regularized-Leader in Two-Player Zero-Sum Markov Games$ $O(T^{-1})$ Convergence of Optimistic-Follow-the-Regularized-Leader in Two-Player Zero-Sum Markov Games Yuepeng Yang Cong Ma 37 14 0 26 Sep 2022
Minimax-Optimal Multi-Agent RL in Markov Games With a Generative Model Gen Li Yuejie Chi Yuting Wei Yuxin Chen 32 18 0 22 Aug 2022
Last-Iterate Convergence with Full and Noisy Feedback in Two-Player Zero-Sum Games Kenshi Abe Kaito Ariu Mitsuki Sakamoto Kenta Toyoshima Atsushi Iwasaki 28 11 0 21 Aug 2022
Efficiently Computing Nash Equilibria in Adversarial Team Markov Games Fivos Kalogiannis Ioannis Anagnostides Ioannis Panageas Emmanouil-Vasileios Vlatakis-Gkaragkounis Vaggos Chatziafratis S. Stavroulakis 33 13 0 03 Aug 2022
Regret Minimization and Convergence to Equilibria in General-sum Markov Games Liad Erez Tal Lancewicki Uri Sherman Tomer Koren Yishay Mansour 40 25 0 28 Jul 2022
Optimism in Face of a Context: Regret Guarantees for Stochastic Contextual MDP Orin Levy Yishay Mansour 13 7 0 22 Jul 2022
Fast Convergence of Optimistic Gradient Ascent in Network Zero-Sum Extensive Form Games Georgios Piliouras Lillian J. Ratliff Ryann Sim Stratis Skoulakis MLT 11 3 0 18 Jul 2022
A Survey of Decision Making in Adversarial Games Xiuxian Li Min Meng Yiguang Hong Jie-bin Chen AAML 29 12 0 16 Jul 2022
Policy Optimization for Markov Games: Unified Framework and Faster Convergence Runyu Zhang Qinghua Liu Haiquan Wang Caiming Xiong Na Li Yu Bai 21 26 0 06 Jun 2022
Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games Sihan Zeng Thinh T. Doan J. Romberg 60 22 0 27 May 2022
Tight Last-Iterate Convergence of the Extragradient and the Optimistic Gradient Descent-Ascent Algorithm for Constrained Monotone Variational Inequalities Yang Cai Argyris Oikonomou Weiqiang Zheng 19 15 0 20 Apr 2022
Independent Natural Policy Gradient Methods for Potential Games: Finite-time Global Convergence with Entropy Regularization Shicong Cen Fan Chen Yuejie Chi 33 15 0 12 Apr 2022
Finite-Time Analysis of Natural Actor-Critic for POMDPs Semih Cayci Niao He R. Srikant 22 1 0 20 Feb 2022
Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergence Dongsheng Ding Chen-Yu Wei Kaipeng Zhang M. Jovanović 22 69 0 08 Feb 2022
Near-Optimal Learning of Extensive-Form Games with Imperfect Information Yunru Bai Chi Jin Song Mei Tiancheng Yu 21 26 0 03 Feb 2022
Finite-Sample Analysis of Decentralized Q-Learning for Stochastic Games Zuguang Gao Qianqian Ma Tamer Bacsar J. Birge OffRL 22 7 0 15 Dec 2021
Doubly Optimal No-Regret Online Learning in Strongly Monotone Games with Bandit Feedback Wenjia Ba Tianyi Lin Jiawei Zhang Zhengyuan Zhou 19 9 0 06 Dec 2021
MDPGT: Momentum-based Decentralized Policy Gradient Tracking Zhanhong Jiang Xian Yeow Lee Sin Yong Tan Kai Liang Tan Aditya Balu Young M. Lee C. Hegde S. Sarkar 8 7 0 06 Dec 2021
Independent Learning in Stochastic Games Asuman Ozdaglar M. O. Sayin Kaipeng Zhang 16 22 0 23 Nov 2021
V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RL Chi Jin Qinghua Liu Yuanhao Wang Tiancheng Yu OffRL 26 131 0 27 Oct 2021
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning Weichao Mao Lin F. Yang Kaipeng Zhang Tamer Bacsar 33 57 0 12 Oct 2021
Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Games Weichao Mao Tamer Basar 23 66 0 12 Oct 2021