Independent Natural Policy Gradient Always Converges in Markov Potential Games

20 October 2021

Roy Fox

Stephen Marcus McAleer

W. Overman

Ioannis Panageas

ArXiv PDF HTML

Papers citing "Independent Natural Policy Gradient Always Converges in Markov Potential Games"

29 / 29 papers shown

Title
Simultaneous Best-Response Dynamics in Random Potential Games Galit Ashkenazi-Golan Domenico Mergoni Cecchelli Edward Plumb 21 0 0 15 May 2025
Independent Learning in Performative Markov Potential Games Rilind Sahitaj Paulius Sasnauskas Yiğit Yalın Debmalya Mandal Goran Radanović 36 0 0 29 Apr 2025
Independent Policy Mirror Descent for Markov Potential Games: Scaling to Large Number of Players Pragnya Alatur Anas Barakat Niao He 27 2 0 15 Aug 2024
Risk Sensitivity in Markov Games and Multi-Agent Reinforcement Learning: A Systematic Review Hafez Ghaemi Shirin Jamshidi Mohammad Mashreghi M. N. Ahmadabadi Hamed Kebriaei 42 1 0 10 Jun 2024
Beyond Theorems: A Counterexample to Potential Markov Game Criteria Fatemeh Fardno S. Zahedi 52 0 0 13 May 2024
Independent Learning in Constrained Markov Potential Games Philip Jordan Anas Barakat Niao He 41 8 0 27 Feb 2024
Risk-Sensitive Multi-Agent Reinforcement Learning in Network Aggregative Markov Games Hafez Ghaemi Hamed Kebriaei Alireza Ramezani Moghaddam Majid Nili Ahamadabadi 32 1 0 08 Feb 2024
Optimistic Policy Gradient in Multi-Player Markov Games with a Single Controller: Convergence Beyond the Minty Property Ioannis Anagnostides Ioannis Panageas Gabriele Farina T. Sandholm 33 3 0 19 Dec 2023
Provably Fast Convergence of Independent Natural Policy Gradient for Markov Potential Games Youbang Sun Tao-Wen Liu Ruida Zhou P. R. Kumar Shahin Shahrampour 33 11 0 15 Oct 2023
Provably Learning Nash Policies in Constrained Markov Potential Games Pragnya Alatur Giorgia Ramponi Niao He Andreas Krause 29 10 0 13 Jun 2023
Context-Aware Bayesian Network Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning Dingyang Chen Qi Zhang 24 2 0 02 Jun 2023
Zero-sum Polymatrix Markov Games: Equilibrium Collapse and Efficient Computation of Nash Equilibria Fivos Kalogiannis Ioannis Panageas 32 8 0 23 May 2023
Local Optimization Achieves Global Optimality in Multi-Agent Reinforcement Learning Yulai Zhao Zhuoran Yang Zhaoran Wang Jason D. Lee 43 3 0 08 May 2023
Convergence Rates for Localized Actor-Critic in Networked Markov Potential Games Zhaoyi Zhou Zaiwei Chen Yiheng Lin Adam Wierman 46 7 0 08 Mar 2023
Population-size-Aware Policy Optimization for Mean-Field Games Pengdeng Li Xinrun Wang Shuxin Li Hau Chan Bo An 21 2 0 07 Feb 2023
Conjugate Natural Selection Reilly P. Raab Luca de Alfaro Yang Liu 18 0 0 29 Aug 2022
Efficiently Computing Nash Equilibria in Adversarial Team Markov Games Fivos Kalogiannis Ioannis Anagnostides Ioannis Panageas Emmanouil-Vasileios Vlatakis-Gkaragkounis Vaggos Chatziafratis S. Stavroulakis 39 13 0 03 Aug 2022
Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Games Stephen Marcus McAleer JB Lanier Kevin A. Wang Pierre Baldi Roy Fox T. Sandholm 35 18 0 13 Jul 2022
Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games Dingyang Chen Qi Zhang Thinh T. Doan 26 12 0 15 Jun 2022
ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret Stephen Marcus McAleer Gabriele Farina Marc Lanctot T. Sandholm 32 24 0 08 Jun 2022
Learning in Congestion Games with Bandit Feedback Qiwen Cui Zhihan Xiong Maryam Fazel S. Du 26 12 0 04 Jun 2022
Learning Distributed and Fair Policies for Network Load Balancing as Markov Potential Game Zhiyuan Yao Zihan Ding OffRL 24 2 0 03 Jun 2022
Independent Natural Policy Gradient Methods for Potential Games: Finite-time Global Convergence with Entropy Regularization Shicong Cen Fan Chen Yuejie Chi 33 15 0 12 Apr 2022
The Complexity of Markov Equilibrium in Stochastic Games C. Daskalakis Noah Golowich Kaipeng Zhang 36 57 0 08 Apr 2022
Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergence Dongsheng Ding Chen-Yu Wei Kaipeng Zhang M. Jovanović 22 69 0 08 Feb 2022
On the Global Convergence Rates of Decentralized Softmax Gradient Play in Markov Potential Games Runyu Zhang Jincheng Mei Bo Dai Dale Schuurmans Na Li 31 20 0 02 Feb 2022
Decentralized Cooperative Reinforcement Learning with Hierarchical Information Structure Hsu Kao Chen-Yu Wei V. Subramanian 15 12 0 01 Nov 2021
On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning Weichao Mao Lin F. Yang Kaipeng Zhang Tamer Bacsar 39 57 0 12 Oct 2021
Independent Policy Gradient Methods for Competitive Reinforcement Learning C. Daskalakis Dylan J. Foster Noah Golowich 64 158 0 11 Jan 2021