Direct Preference Optimization with an Offset

16 February 2024

Papers citing "Direct Preference Optimization with an Offset"

14 / 14 papers shown

Title
Design Considerations in Offline Preference-based RL Alekh Agarwal Christoph Dann T. V. Marinov OffRL 76 0 0 08 Feb 2025
Understanding the Logic of Direct Preference Alignment through Logic Kyle Richardson Vivek Srikumar Ashish Sabharwal 140 2 0 23 Dec 2024
Sharp Analysis for KL-Regularized Contextual Bandits and RLHF Heyang Zhao Chenlu Ye Quanquan Gu Tong Zhang OffRL 164 6 0 07 Nov 2024
TODO: Enhancing LLM Alignment with Ternary Preferences Yuxiang Guo Lu Yin Bo Jiang Jiaqi Zhang 92 1 0 02 Nov 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 382 2 0 11 Oct 2024
Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift Seongho Son William Bankes Sayak Ray Chowdhury Brooks Paige Ilija Bogunovic 73 4 0 26 Jul 2024
3D-Properties: Identifying Challenges in DPO and Charting a Path Forward Yuzi Yan Yibo Miao J. Li Yipin Zhang Jian Xie Zhijie Deng Dong Yan 71 12 0 11 Jun 2024
Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization Hritik Bansal Ashima Suvarna Gantavya Bhatt Nanyun Peng Kai-Wei Chang Aditya Grover ALM 95 10 0 31 Mar 2024
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 155 597 0 18 Oct 2023
Quark: Controllable Text Generation with Reinforced Unlearning Ximing Lu Sean Welleck Jack Hessel Liwei Jiang Lianhui Qin Peter West Prithviraj Ammanabrolu Yejin Choi MU 99 213 0 26 May 2022
Recursively Summarizing Books with Human Feedback Jeff Wu Long Ouyang Daniel M. Ziegler Nissan Stiennon Ryan J. Lowe Jan Leike Paul Christiano ALM 93 301 0 22 Sep 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 434 1,664 0 18 Sep 2019
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 285 18,685 0 20 Jul 2017
Asynchronous Methods for Deep Reinforcement Learning Volodymyr Mnih Adria Puigdomenech Badia M. Berk Mirza Alex Graves Timothy Lillicrap Tim Harley David Silver Koray Kavukcuoglu 170 8,805 0 04 Feb 2016