Accelerating Nash Learning from Human Feedback via Mirror Prox

Accelerating Nash Learning from Human Feedback via Mirror Prox

26 May 2025

Daniele Calandriello

Denis Belomestny

Papers citing "Accelerating Nash Learning from Human Feedback via Mirror Prox"

11 / 11 papers shown

Title
Improving LLM General Preference Alignment via Optimistic Online Mirror Descent Yuheng Zhang Dian Yu Tao Ge Linfeng Song Zhichen Zeng Haitao Mi Nan Jiang Dong Yu 86 4 0 24 Feb 2025
Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment Mingzhi Wang Chengdong Ma Qizhi Chen Linjian Meng Yang Han Jiancong Xiao Zhaowei Zhang Jing Huo Weijie Su Yaodong Yang 89 7 0 22 Oct 2024
RRM: Robust Reward Model Training Mitigates Reward Hacking Tianqi Liu Wei Xiong Jie Jessie Ren Lichang Chen Junru Wu ... Yuan Liu Bilal Piot Abe Ittycheriah Aviral Kumar Mohammad Saleh AAML 58 18 0 20 Sep 2024
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 81 580 0 18 Oct 2023
ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret Stephen Marcus McAleer Gabriele Farina Marc Lanctot Tuomas Sandholm 55 25 0 08 Jun 2022
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 31 42,038 0 03 Dec 2019
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 379 1,664 0 18 Sep 2019
On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift Alekh Agarwal Sham Kakade Jason D. Lee G. Mahajan 23 320 0 01 Aug 2019
Preference-based Online Learning with Dueling Bandits: A Survey Viktor Bengs R. Busa-Fekete Adil El Mesaoudi-Paul Eyke Hüllermeier 47 112 0 30 Jul 2018
Continuous control with deep reinforcement learning Timothy Lillicrap Jonathan J. Hunt Alexander Pritzel N. Heess Tom Erez Yuval Tassa David Silver Daan Wierstra 104 13,174 0 09 Sep 2015
Optimization, Learning, and Games with Predictable Sequences Alexander Rakhlin Karthik Sridharan 30 377 0 08 Nov 2013