Visualising Policy-Reward Interplay to Inform Zeroth-Order Preference Optimisation of Large Language Models

5 March 2025

Papers citing "Visualising Policy-Reward Interplay to Inform Zeroth-Order Preference Optimisation of Large Language Models"

11 / 11 papers shown

Title
Direct Language Model Alignment from Online AI Feedback Shangmin Guo Biao Zhang Tianlin Liu Tianqi Liu Misha Khalman ... Thomas Mesnard Yao-Min Zhao Bilal Piot Johan Ferret Mathieu Blondel ALM 99 160 0 07 Feb 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 167 1,288 0 05 Feb 2024
Secrets of RLHF in Large Language Models Part II: Reward Modeling Bing Wang Rui Zheng Luyao Chen Yan Liu Shihan Dou ... Qi Zhang Xipeng Qiu Xuanjing Huang Zuxuan Wu Yuanyuan Jiang ALM 113 110 0 11 Jan 2024
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 180 647 0 18 Oct 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 389 4,163 0 29 May 2023
Fine-Tuning Language Models with Just Forward Passes Sadhika Malladi Tianyu Gao Eshaan Nichani Alexandru Damian Jason D. Lee Danqi Chen Sanjeev Arora 128 206 0 27 May 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 888 13,207 0 04 Mar 2022
Gradient Surgery for Multi-Task Learning Tianhe Yu Saurabh Kumar Abhishek Gupta Sergey Levine Karol Hausman Chelsea Finn 191 1,230 0 19 Jan 2020
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 491 1,768 0 18 Sep 2019
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 568 19,296 0 20 Jul 2017
Deep reinforcement learning from human preferences Paul Christiano Jan Leike Tom B. Brown Miljan Martic Shane Legg Dario Amodei 218 3,377 0 12 Jun 2017