Understanding Reference Policies in Direct Preference Optimization

Understanding Reference Policies in Direct Preference Optimization

18 July 2024

Arman Cohan

Papers citing "Understanding Reference Policies in Direct Preference Optimization"

9 / 9 papers shown

Title
Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model Junshu Pan Wei Shen Shulin Huang Qiji Zhou Yue Zhang 71 0 0 22 Apr 2025
Reasoning Beyond Limits: Advances and Open Problems for LLMs M. Ferrag Norbert Tihanyi Merouane Debbah ELM OffRL LRM AI4CE 131 2 0 26 Mar 2025
SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters Teng Xiao Yige Yuan Z. Chen Mingxiao Li Shangsong Liang Z. Ren V. Honavar 95 5 0 21 Feb 2025
Video-Text Dataset Construction from Multi-AI Feedback: Promoting Weak-to-Strong Preference Learning for Video Large Language Models Hao Yi Qingyang Li Yihan Hu Fuzheng Zhang Di Zhang Yong Liu VGen 71 0 0 25 Nov 2024
RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization Hanyang Zhao Genta Indra Winata Anirban Das Shi-Xiong Zhang D. Yao Wenpin Tang Sambit Sahu 54 5 0 05 Oct 2024
Modulated Intervention Preference Optimization (MIPO): Keep the Easy, Refine the Difficult Cheolhun Jang 28 0 0 26 Sep 2024
Preference Learning Algorithms Do Not Learn Preference Rankings Angelica Chen Sadhika Malladi Lily H. Zhang Xinyi Chen Qiuyi Zhang Rajesh Ranganath Kyunghyun Cho 30 23 0 29 May 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Classical Structured Prediction Losses for Sequence to Sequence Learning Sergey Edunov Myle Ott Michael Auli David Grangier MarcÁurelio Ranzato AIMat 56 185 0 14 Nov 2017