PIPA: Preference Alignment as Prior-Informed Statistical Estimation

9 February 2025

Papers citing "PIPA: Preference Alignment as Prior-Informed Statistical Estimation"

26 / 26 papers shown

Title
Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMs Yuzhe Gu Wentao Zhang Chengqi Lyu Dahua Lin Kai Chen 106 2 0 04 Mar 2025
UI-TARS: Pioneering Automated GUI Interaction with Native Agents Yujia Qin Yining Ye Junjie Fang Han Wang Shihao Liang ... Haifeng Liu F. Lin Tao Peng Xin Liu Guang Shi LLMAG LM&Ro 98 69 0 21 Jan 2025
Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback Yen-Ting Lin Di Jin Tengyu Xu Tianhao Wu Sainbayar Sukhbaatar ... Yuandong Tian Arash Rahnama Sinong Wang Hao Ma Han Fang LRM 51 5 0 18 Jan 2025
rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking Xinyu Guan Lefei Zhang Yifei Liu Ning Shang Youran Sun Yi Zhu Fan Yang Mao Yang LRM SyDa ReLM 120 133 0 08 Jan 2025
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 196 35 0 11 Oct 2024
Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization Guanlin Liu Kaixuan Ji Ning Dai Zheng Wu Chen Dun Q. Gu Lin Yan Quanquan Gu Lin Yan OffRL LRM 132 13 0 11 Oct 2024
Latent Feature Mining for Predictive Model Enhancement with Large Language Models Bingxuan Li Pengyi Shi Amy Ward 119 0 0 06 Oct 2024
Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs Xin Lai Zhuotao Tian Yukang Chen Senqiao Yang Xiangru Peng Jiaya Jia LRM 153 126 0 26 Jun 2024
Step-level Value Preference Optimization for Mathematical Reasoning Guoxin Chen Minpeng Liao Chengxi Li Kai Fan LRM 88 42 0 16 Jun 2024
OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework Jian Hu Xibin Wu Weixun Wang OpenLLMAI Team Dehao Zhang Yu Cao AI4CE VLM 106 130 0 20 May 2024
Iterative Reasoning Preference Optimization Richard Yuanzhe Pang Weizhe Yuan Kyunghyun Cho He He Sainbayar Sukhbaatar Jason Weston LRM 127 137 0 30 Apr 2024
DPO Meets PPO: Reinforced Token Optimization for RLHF Han Zhong Zikang Shan Guhao Feng Wei Xiong Xinle Cheng Li Zhao Di He Jiang Bian Liwei Wang 125 72 0 29 Apr 2024
Token-level Direct Preference Optimization Yongcheng Zeng Guoqing Liu Weiyu Ma Ning Yang Haifeng Zhang Jun Wang 106 64 0 18 Apr 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 167 1,288 0 05 Feb 2024
BRAIn: Bayesian Reward-conditioned Amortized Inference for natural language generation from feedback Gaurav Pandey Yatin Nandwani Tahira Naseem Mayank Mishra Guangxuan Xu Dinesh Raghu Sachindra Joshi Asim Munawar Ramón Fernández Astudillo BDL 66 4 0 04 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 275 569 0 02 Feb 2024
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 180 647 0 18 Oct 2023
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 198 1,240 0 31 May 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 389 4,163 0 29 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,761 0 15 Mar 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 888 13,207 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 850 9,714 0 28 Jan 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 353 4,598 0 27 Oct 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 502 10,526 0 17 Jun 2021
Distributional Reinforcement Learning for Energy-Based Sequential Models Tetiana Parshakova J. Andreoli Marc Dymetman 72 21 0 18 Dec 2019
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 553 19,296 0 20 Jul 2017