Direct Preference Optimization With Unobserved Preference Heterogeneity

23 May 2024

Papers citing "Direct Preference Optimization With Unobserved Preference Heterogeneity"

17 / 17 papers shown

Title
Advances in Preference-based Reinforcement Learning: A Review Youssef Abdelkareem Shady Shehata Fakhri Karray OffRL 66 10 0 21 Aug 2024
RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation Chanwoo Park Mingyang Liu Dingwen Kong Kaiqing Zhang Asuman Ozdaglar 80 38 0 30 Apr 2024
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards Haoxiang Wang Yong Lin Wei Xiong Rui Yang Shizhe Diao Shuang Qiu Han Zhao Tong Zhang 84 83 0 28 Feb 2024
Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF Anand Siththaranjan Cassidy Laidlaw Dylan Hadfield-Menell 79 66 0 13 Dec 2023
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 163 615 0 18 Oct 2023
Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization Zhanhui Zhou Jie Liu Chao Yang Jing Shao Yu Liu Xiangyu Yue Wanli Ouyang Yu Qiao 68 60 0 05 Oct 2023
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards Alexandre Ramé Guillaume Couairon Mustafa Shukor Corentin Dancette Jean-Baptiste Gaya Laure Soulier Matthieu Cord MoMe 53 149 0 07 Jun 2023
Settling the Reward Hypothesis Michael Bowling John D. Martin David Abel Will Dabney LRM 67 33 0 20 Dec 2022
On the Sensitivity of Reward Inference to Misspecified Human Models Joey Hong Kush S. Bhatia Anca Dragan 45 26 0 09 Dec 2022
Fine-tuning language models to find agreement among humans with diverse preferences Michiel A. Bakker Martin Chadwick Hannah R. Sheahan Michael Henry Tessler Lucy Campbell-Gillingham ... Nat McAleese Amelia Glaese John Aslanides M. Botvinick Christopher Summerfield ALM 100 230 0 28 Nov 2022
A General Language Assistant as a Laboratory for Alignment Amanda Askell Yuntao Bai Anna Chen Dawn Drain Deep Ganguli ... Tom B. Brown Jack Clark Sam McCandlish C. Olah Jared Kaplan ALM 114 775 0 01 Dec 2021
B-Pref: Benchmarking Preference-Based Reinforcement Learning Kimin Lee Laura M. Smith Anca Dragan Pieter Abbeel OffRL 73 97 0 04 Nov 2021
Recursively Summarizing Books with Human Feedback Jeff Wu Long Ouyang Daniel M. Ziegler Nissan Stiennon Ryan J. Lowe Jan Leike Paul Christiano ALM 139 302 0 22 Sep 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 452 1,717 0 18 Sep 2019
Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback Khanh Nguyen Hal Daumé Jordan L. Boyd-Graber 62 138 0 24 Jul 2017
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 444 18,931 0 20 Jul 2017
Optimization, Learning, and Games with Predictable Sequences Alexander Rakhlin Karthik Sridharan 89 379 0 08 Nov 2013