v1v2 (latest)

SimPO: Simple Preference Optimization with a Reference-Free Reward

23 May 2024

Papers citing "SimPO: Simple Preference Optimization with a Reference-Free Reward"

50 / 197 papers shown

Title
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models Yang Sui Yu-Neng Chuang Guanchu Wang Jiamu Zhang Tianyi Zhang ... Hongyi Liu Andrew Wen Shaochen Zhong Hanjie Chen OffRL ReLM LRM 204 101 0 20 Mar 2025
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment Jia-Nan Li Jian Guan Songhao Wu Wei Wu Rui Yan 169 3 0 19 Mar 2025
Aligning Multimodal LLM with Human Preference: A Survey Tao Yu Yize Zhang Chaoyou Fu Junkang Wu Jinda Lu ... Qingsong Wen Zheng Zhang Yan Huang Liang Wang Tieniu Tan 439 4 0 18 Mar 2025
Fine-Tuning Diffusion Generative Models via Rich Preference Optimization Hanyang Zhao Haoxian Chen Yucheng Guo Genta Indra Winata Tingting Ou Ziyu Huang D. Yao Wenpin Tang 130 0 0 13 Mar 2025
Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning Bowen Jin Hansi Zeng Zhenrui Yue Dong Wang Sercan O. Arik Dong Wang Hamed Zamani Jiawei Han RALM ReLM KELM OffRL AI4TS LRM 226 122 0 12 Mar 2025
Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis Letian Zhang Quan Cui Bingchen Zhao Cheng Yang MLLM SyDa 112 1 0 11 Mar 2025
DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs Jongwoo Ko Tianyi Chen Sungnyun Kim Tianyu Ding Luming Liang Ilya Zharkov Se-Young Yun VLM 458 2 0 10 Mar 2025
DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models Ruizhe Chen Wenhao Chai Zhifei Yang Xiaotian Zhang Qiufeng Wang Tony Q.S. Quek Soujanya Poria Zuozhu Liu 133 1 0 06 Mar 2025
SOLAR: Scalable Optimization of Large-scale Architecture for Reasoning Chen Li Yinyi Luo Anudeep Bolimera Uzair Ahmed Siyang Song Hrishikesh Gokhale Marios Savvides LRM AI4CE 123 1 0 06 Mar 2025
Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment Wen Yang Junhong Wu Chen Wang Chengqing Zong J.N. Zhang 159 1 0 06 Mar 2025
DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models Yi Shen Jing Zhang Jieyun Huang Shuming Shi Wenjing Zhang Jiangze Yan Rongjia Du Ning Wang Kai Wang Shiguo Lian LRM 132 54 0 06 Mar 2025
Preserving Cultural Identity with Context-Aware Translation Through Multi-Agent AI Systems Mahfuz Ahmed Anik Abdur Rahman Azmine Toushik Wasi Md Manjurul Ahsan 94 5 0 05 Mar 2025
AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation Songming Zhang Xue Zhang Tong Zhang Bojie Hu Yufeng Chen Jinan Xu 108 1 0 04 Mar 2025
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning Zhibin Lan Liqiang Niu Fandong Meng Jie Zhou Jinsong Su VLM 109 0 0 04 Mar 2025
Kanana: Compute-efficient Bilingual Language Models Kanana LLM Team Yunju Bak Hojin Lee Minho Ryu Jiyeon Ham ... Daniel Lee Minchul Lee MinHyung Lee Shinbok Lee Gaeun Seo 177 1 0 26 Feb 2025
Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMs Zhaowei Zhang Fengshuo Bai Qizhi Chen Chengdong Ma Mingzhi Wang Haoran Sun Zilong Zheng Yaodong Yang 173 5 0 26 Feb 2025
Preference-Based Gradient Estimation for ML-Guided Approximate Combinatorial Optimization Arman Mielke Uwe Bauknecht Thilo Strauss Mathias Niepert 154 0 0 26 Feb 2025
Stackelberg Game Preference Optimization for Data-Efficient Alignment of Language Models Xu Chu Zhixin Zhang Tianyu Jia Yujie Jin 141 0 0 25 Feb 2025
AMPO: Active Multi-Preference Optimization for Self-play Preference Selection Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 108 0 0 25 Feb 2025
Advantage-Guided Distillation for Preference Alignment in Small Language Models Shiping Gao Fanqi Wan Jiajian Guo Xiaojun Quan Qifan Wang ALM 150 0 0 25 Feb 2025
Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance Chenghua Huang Lu Wang Fangkai Yang Pu Zhao Zechao Li Qingwei Lin Dongmei Zhang Saravan Rajmohan Qi Zhang OffRL 80 1 0 24 Feb 2025
CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale Chenlong Wang Zhaoyang Chu Zhengxiang Cheng Xuyi Yang Kaiyue Qiu Yao Wan Zhou Zhao Xuanhua Shi Danny Chen ALM SyDa 101 0 0 23 Feb 2025
SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters Teng Xiao Yige Yuan Ziyang Chen Mingxiao Li Shangsong Liang Zhaochun Ren V. Honavar 264 11 0 21 Feb 2025
BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignment Sizhe Wang Yongqi Tong Hengyuan Zhang Dawei Li Xin Zhang Tianlong Chen 214 10 0 21 Feb 2025
Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF Shicong Cen Jincheng Mei Katayoon Goshvadi Hanjun Dai Tong Yang Sherry Yang Dale Schuurmans Yuejie Chi Bo Dai OffRL 150 37 0 20 Feb 2025
Faster WIND: Accelerating Iterative Best-of- $N$ Distillation for LLM Alignment Tong Yang Jincheng Mei H. Dai Zixin Wen Shicong Cen Dale Schuurmans Yuejie Chi Bo Dai 120 4 0 20 Feb 2025
Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guarantees Yongtao Wu Luca Viano Yihang Chen Zhenyu Zhu Kimon Antonakopoulos Quanquan Gu Volkan Cevher 175 1 0 18 Feb 2025
BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop Lucas Charpentier Leshem Choshen Ryan Cotterell Mustafa Omer Gul Michael Y. Hu ... Candace Ross Raj Sanjay Shah Alex Warstadt Ethan Gotlieb Wilcox Adina Williams 122 5 0 15 Feb 2025
PIPA: Preference Alignment as Prior-Informed Statistical Estimation Junbo Li Zhangyang Wang Qiang Liu OffRL 196 0 0 09 Feb 2025
Design Considerations in Offline Preference-based RL Alekh Agarwal Christoph Dann T. V. Marinov OffRL 108 1 0 08 Feb 2025
Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial? Wenzhe Li Yong Lin Mengzhou Xia Chi Jin MoE 148 4 0 02 Feb 2025
Diverse Preference Optimization Jack Lanchantin Angelica Chen Shehzaad Dhuliawala Ping Yu Jason Weston Sainbayar Sukhbaatar Ilia Kulikov 240 4 0 30 Jan 2025
Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning Nirav Diwan Tolga Ergen Dongsub Shim Honglak Lee 89 0 0 28 Jan 2025
LiPO: Listwise Preference Optimization through Learning-to-Rank Tianqi Liu Zhen Qin Junru Wu Jiaming Shen Misha Khalman ... Mohammad Saleh Simon Baumgartner Jialu Liu Peter J. Liu Xuanhui Wang 328 60 0 28 Jan 2025
Controllable Protein Sequence Generation with LLM Preference Optimization Xiangyu Liu Yi Liu Silei Chen Wei Hu 111 1 0 28 Jan 2025
Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback Yafu Li Xuyang Hu Xiaoye Qu Linjie Li Yu Cheng 122 8 0 22 Jan 2025
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning Haotian Luo Li Shen Haiying He Yun Wang Shiwei Liu Wei Li Naiqiang Tan Xiaochun Cao Dacheng Tao VLM LRM 164 97 0 22 Jan 2025
FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings Tong Liu Xiao Yu Wenxuan Zhou Jindong Gu Volker Tresp 82 1 0 11 Jan 2025
AlignAb: Pareto-Optimal Energy Alignment for Designing Nature-Like Antibodies Yibo Wen Chenwei Xu Jerry Yao-Chieh Hu Han Liu DiffM 104 5 0 31 Dec 2024
Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs Xingyu Chen Jiahao Xu Tian Liang Zhiwei He Jianhui Pang ... Zizhuo Zhang Rui Wang Zhaopeng Tu Haitao Mi Dong Yu LRM ReLM 206 197 0 30 Dec 2024
Understanding the Logic of Direct Preference Alignment through Logic Kyle Richardson Vivek Srikumar Ashish Sabharwal 222 2 0 23 Dec 2024
REFA: Reference Free Alignment for multi-preference optimization Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 182 1 0 20 Dec 2024
Adapting While Learning: Grounding LLMs for Scientific Problems with Intelligent Tool Usage Adaptation Bohan Lyu Yadi Cao Duncan Watson-Parris Leon Bergen Taylor Berg-Kirkpatrick Rose Yu 131 5 0 01 Nov 2024
Constraint Back-translation Improves Complex Instruction Following of Large Language Models Yunjia Qi Hao Peng Xinyu Wang Bin Xu Lei Hou Juanzi Li 106 4 0 31 Oct 2024
$f$ -PO: Generalizing Preference Optimization with $f$ -divergence Minimization Jiaqi Han Mingjian Jiang Yuxuan Song J. Leskovec Stefano Ermon 121 6 0 29 Oct 2024
Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model Wenhong Zhu Zhiwei He Xiaofeng Wang Pengfei Liu Rui Wang OSLM 109 6 0 24 Oct 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 137 6 0 23 Oct 2024
Modality-Fair Preference Optimization for Trustworthy MLLM Alignment Songtao Jiang Yan Zhang Ruizhe Chen Yeying Jin Zuozhu Liu Qinglin He Yang Feng Jian Wu Zuozhu Liu MoE MLLM 100 12 0 20 Oct 2024
Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning H. Fernando Han Shen Parikshit Ram Yi Zhou Horst Samulowitz Nathalie Baracaldo Tianyi Chen CLL 169 4 0 20 Oct 2024
Preference Diffusion for Recommendation Shuo Liu An Zhang Guoqing Hu Hong Qian Tat-Seng Chua 160 1 0 17 Oct 2024