Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

11 October 2024

Sadhika Malladi

Adithya Bhaskar

Papers citing "Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization"

12 / 12 papers shown

Title
SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment Wenqiao Zhu Ji Liu Lulu Wang Jun Wu Yulun Zhang 7 0 0 18 May 2025
Spectral Policy Optimization: Coloring your Incorrect Reasoning in GRPO Peter Chen Xiaopeng Li Zhiyu Li Xi Chen Tianyi Lin 9 0 0 16 May 2025
ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via $α$ - $β$ -Divergence Guanghui Wang Zhiyong Yang Zihan Wang Shi Wang Qianqian Xu Qingming Huang 42 0 0 07 May 2025
SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning Tianjian Li Daniel Khashabi 55 0 0 05 May 2025
Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models Wei Chen Xin Yan Bin Wen Fan Yang Tingting Gao Di Zhang Long Chen MLLM 97 0 0 09 Apr 2025
Conformal Linguistic Calibration: Trading-off between Factuality and Specificity Zhengping Jiang Anqi Liu Benjamin Van Durme 92 1 0 26 Feb 2025
Is Free Self-Alignment Possible? Dyah Adila Changho Shin Yijing Zhang Frederic Sala MoMe 118 2 0 24 Feb 2025
Training a Generally Curious Agent Fahim Tajwar Yiding Jiang Abitha Thankaraj Sumaita Sadia Rahman J. Zico Kolter Jeff Schneider Ruslan Salakhutdinov 123 1 0 24 Feb 2025
Preference learning made easy: Everything should be understood through win rate Lily H. Zhang Rajesh Ranganath 87 0 0 14 Feb 2025
PIPA: Preference Alignment as Prior-Informed Statistical Estimation Junbo Li Zhangyang Wang Qiang Liu OffRL 104 0 0 09 Feb 2025
Understanding the Logic of Direct Preference Alignment through Logic Kyle Richardson Vivek Srikumar Ashish Sabharwal 85 2 0 23 Dec 2024
Robust Preference Optimization through Reward Model Distillation Adam Fisch Jacob Eisenstein Vicky Zayats Alekh Agarwal Ahmad Beirami Chirag Nagpal Peter Shaw Jonathan Berant 81 22 0 29 May 2024