Direct Preference Optimization: Your Language Model is Secretly a Reward Model

29 May 2023

Christopher D. Manning

Chelsea Finn

ALM

ArXiv PDF HTML

Papers citing "Direct Preference Optimization: Your Language Model is Secretly a Reward Model"

50 / 2,637 papers shown

Title
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs Ling Team B. Zeng Chenyu Huang Chao Zhang Changxin Tian ... Zhaoxin Huan Zujie Wen Zhenhang Sun Zhuoxuan Du Z. He MoE ALM 111 2 0 07 Mar 2025
Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentation Aishik Konwer Zhijian Yang Erhan Bas Cao Xiao Prateek Prasanna Parminder Bhatia Taha A. Kass-Hout MedIm VLM 78 0 0 06 Mar 2025
DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models Ruizhe Chen Wenhao Chai Zhifei Yang Xiaotian Zhang Qiufeng Wang Tony Q.S. Quek Soujanya Poria Zuozhu Liu 55 0 0 06 Mar 2025
Integrating Protein Dynamics into Structure-Based Drug Design via Full-Atom Stochastic Flows Xiangxin Zhou Yi Xiao Haowei Lin Xinheng He Jiaqi Guan Yang Wang Qiang Liu F. I. S. Kevin Zhou Liang Wang Jianzhu Ma AI4CE 60 0 0 06 Mar 2025
Towards Autonomous Reinforcement Learning for Real-World Robotic Manipulation with Large Language Models Niccolò Turcato Matteo Iovino Aris Synodinos Alberto Dalla Libera R. Carli Pietro Falco LM&Ro 48 0 0 06 Mar 2025
SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winner Kejia Chen Jiawen Zhang Jiacong Hu Jiazhen Yang Jian Lou Zunlei Feng Mingli Song 74 0 0 06 Mar 2025
Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment Wen Yang Junhong Wu Chen Wang Chengqing Zong J.N. Zhang 77 1 0 06 Mar 2025
Adding Alignment Control to Language Models Wenhong Zhu Weinan Zhang Rui Wang 65 0 0 06 Mar 2025
ValuePilot: A Two-Phase Framework for Value-Driven Decision-Making Yitong Luo Hou Hei Lam Ziang Chen Zhenliang Zhang Xue Feng 77 0 0 06 Mar 2025
SOLAR: Scalable Optimization of Large-scale Architecture for Reasoning Chen Li Yinyi Luo Anudeep Bolimera Uzair Ahmed Shri Kiran Srinivasan Hrishikesh Gokhale Marios Savvides LRM AI4CE 68 1 0 06 Mar 2025
UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets Wenyu Wang Hao Fei Xiaotian Ye Z. Z. Ren Ziyang Chen Pengjie Ren MU KELM 227 0 0 06 Mar 2025
Uncovering Gaps in How Humans and LLMs Interpret Subjective Language Erik Jones Arjun Patrawala Jacob Steinhardt 49 0 0 06 Mar 2025
Shifting Long-Context LLMs Research from Input to Output Yuhao Wu Yushi Bai Zhiqing Hu Shangqing Tu Ming Shan Hee Juanzi Li Roy Ka-wei Lee 65 1 0 06 Mar 2025
DB-Explore: Automated Database Exploration and Instruction Synthesis for Text-to-SQL Haoyuan Ma Yongliang Shen Hengwei Liu Wenqi Zhang Haolei Xu Qiuying Peng Jun Wang Weiming Lu 54 2 0 06 Mar 2025
Improving LLM Safety Alignment with Dual-Objective Optimization Xuandong Zhao Will Cai Tianneng Shi David Huang Licong Lin Song Mei Dawn Song AAML MU 72 1 0 05 Mar 2025
Extrapolation Merging: Keep Improving With Extrapolation and Merging Yiguan Lin Bin Xu Yinghao Li Yang Gao MoMe 59 1 0 05 Mar 2025
Preserving Cultural Identity with Context-Aware Translation Through Multi-Agent AI Systems Mahfuz Ahmed Anik Abdur Rahman Azmine Toushik Wasi Md Manjurul Ahsan 49 0 0 05 Mar 2025
Human Preferences for Constructive Interactions in Language Model Alignment Yara Kyrychenko Jon Roozenbeek Brandon Davidson S. V. D. Linden Ramit Debnath 48 0 0 05 Mar 2025
FANS -- Formal Answer Selection for Natural Language Math Reasoning Using Lean4 Jiarui Yao Ruida Wang Tong Zhang LRM 67 0 0 05 Mar 2025
Visualising Policy-Reward Interplay to Inform Zeroth-Order Preference Optimisation of Large Language Models Alessio Galatolo Zhenbang Dai Katie Winkle Meriem Beloucif 60 0 0 05 Mar 2025
Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generation Hiroshi Takahashi Tomoharu Iwata Atsutoshi Kumagai Yuuki Yamanaka Tomoya Yamashita DiffM 72 0 0 05 Mar 2025
Process-based Self-Rewarding Language Models Shimao Zhang Xiao Liu Xin Zhang Junxiao Liu Zheheng Luo Shujian Huang Yeyun Gong ReLM SyDa LRM 95 3 0 05 Mar 2025
AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation Songming Zhang Xue Zhang Tong Zhang Bojie Hu Yufeng Chen Jinan Xu 60 1 0 04 Mar 2025
Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models Joykirat Singh Tanmoy Chakraborty A. Nambi AI4Cl LRM ReLM 60 1 0 04 Mar 2025
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning Zhibin Lan Liqiang Niu Fandong Meng Jie Zhou Jinsong Su VLM 77 2 0 04 Mar 2025
Effectively Steer LLM To Follow Preference via Building Confident Directions Bingqing Song Boran Han Shuai Zhang Hao Wang Haoyang Fang Bonan Min Yuyang Wang Mingyi Hong LLMSV 57 0 0 04 Mar 2025
MPO: Boosting LLM Agents with Meta Plan Optimization Weimin Xiong Yifan Song Qingxiu Dong Bingchan Zhao Feifan Song Xun Wang Sujian Li LLMAG 89 1 0 04 Mar 2025
IterPref: Focal Preference Learning for Code Generation via Iterative Debugging Jie Wu Haoling Li Xin Zhang Jianwen Luo Yangyu Huang Ruihang Chu Yue Yang Scarlett Li 80 0 0 04 Mar 2025
Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMs Yuzhe Gu Feiyu Xiong Chengqi Lyu Dahua Lin Kai Chen 72 1 0 04 Mar 2025
Language Models can Self-Improve at State-Value Estimation for Better Search Ethan Mendes Alan Ritter LRM 68 3 0 04 Mar 2025
Adversarial Tokenization Renato Lui Geh Zilei Shao Guy Van den Broeck SILM AAML 92 0 0 04 Mar 2025
Memorize or Generalize? Evaluating LLM Code Generation with Evolved Questions Wentao Chen Lizhe Zhang Li Zhong Letian Peng Zilong Wang Jingbo Shang ELM 98 1 0 04 Mar 2025
Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models Zhifei Xie Mingbao Lin Ziqiang Liu Pengcheng Wu Shuicheng Yan Chunyan Miao AuLLM OffRL LRM 87 9 0 04 Mar 2025
In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models David Ponce Thierry Etchegoyhen 79 1 0 03 Mar 2025
Jailbreaking Safeguarded Text-to-Image Models via Large Language Models Zhengyuan Jiang Yuepeng Hu Yue Yang Yinzhi Cao Neil Zhenqiang Gong 72 0 0 03 Mar 2025
Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness Tingchen Fu Fazl Barez AAML 67 0 0 03 Mar 2025
VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video Generation Wenhao Wang Yue Yang DiffM VGen 89 0 0 03 Mar 2025
Towards Widening The Distillation Bottleneck for Reasoning Models Huifeng Yin Yu Zhao Mingyang Wu Xuanfan Ni Bo Zeng ... Liangying Shao Chenyang Lyu Longyue Wang Weihua Luo Kaifu Zhang LRM 47 2 0 03 Mar 2025
None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering Zhi Rui Tam Cheng-Kuang Wu Chieh-Yen Lin Yun-Nung Chen 41 1 0 03 Mar 2025
DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models Zhendong Wang Jianmin Bao Shuyang Gu Dong Chen Wengang Zhou Haoyang Li DiffM 53 0 0 03 Mar 2025
Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI Perspective Rakshit Aralimatti Syed Abdul Gaffar Shakhadri Kruthika KR Kartik Basavaraj Angadi 28 0 0 03 Mar 2025
Active Learning for Direct Preference Optimization Branislav Kveton Xintong Li Julian McAuley Ryan Rossi Jingbo Shang Junda Wu Tong Yu 69 1 0 03 Mar 2025
SAKE: Steering Activations for Knowledge Editing Marco Scialanga Thibault Laugel Vincent Grari Marcin Detyniecki KELM LLMSV 83 2 0 03 Mar 2025
CE-U: Cross Entropy Unlearning Bo Yang MU 58 0 0 03 Mar 2025
DPR: Diffusion Preference-based Reward for Offline Reinforcement Learning Teng Pang Bingzheng Wang Guoqiang Wu Yilong Yin OffRL 73 0 0 03 Mar 2025
ReaderLM-v2: Small Language Model for HTML to Markdown and JSON Feng Wang Zesheng Shi Bo Wang Nan Wang Han Xiao RALM 81 1 0 03 Mar 2025
CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdom Yisen Li Lingfeng Yang Wenxuan Shen Pan Zhou Yao Wan Weiwei Lin Danny Chen 80 0 0 03 Mar 2025
Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator Kaiwen Zheng Yongxin Chen Huayu Chen Guande He Xuan Li Jun Zhu Qinsheng Zhang DiffM 49 0 0 03 Mar 2025
What do Large Language Models Say About Animals? Investigating Risks of Animal Harm in Generated Text Arturs Kanepajs Aditi Basu Sankalpa Ghose Constance Li Akshat Mehta Ronak Mehta Samuel David Tucker-Davis Eric Zhou Bob Fischer ALM ELM 50 0 0 03 Mar 2025
Cancer Type, Stage and Prognosis Assessment from Pathology Reports using LLMs Rachit Saluja Jacob Rosenthal Yoav Artzi David J. Pisapia B. Liechty M. Sabuncu LM&MA ELM 70 1 0 03 Mar 2025