v1v2 (latest)

ORPO: Monolithic Preference Optimization without Reference Model

12 March 2024

Papers citing "ORPO: Monolithic Preference Optimization without Reference Model"

50 / 174 papers shown

Title
RePO: ReLU-based Preference Optimization Junkang Wu Kexin Huang Xue Wang Jinyang Gao Bolin Ding Jiancan Wu Xiangnan He Xiang Wang 104 1 0 10 Mar 2025
SOLAR: Scalable Optimization of Large-scale Architecture for Reasoning Chen Li Yinyi Luo Anudeep Bolimera Uzair Ahmed Siyang Song Hrishikesh Gokhale Marios Savvides LRM AI4CE 114 1 0 06 Mar 2025
DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models Ruizhe Chen Wenhao Chai Zhifei Yang Xiaotian Zhang Qiufeng Wang Tony Q.S. Quek Soujanya Poria Zuozhu Liu 133 1 0 06 Mar 2025
Process-based Self-Rewarding Language Models Shimao Zhang Xiao Liu Xin Zhang Junxiao Liu Zheheng Luo Shujian Huang Yeyun Gong ReLM SyDa LRM 133 11 0 05 Mar 2025
Generate, Discriminate, Evolve: Enhancing Context Faithfulness via Fine-Grained Sentence-Level Self-Evolution Keliang Li Tianhua Zhang Yunxiang Li Hongyin Luo Abdalla Moustafa Xixin Wu James Glass Helen Meng 101 2 0 03 Mar 2025
PEO: Improving Bi-Factorial Preference Alignment with Post-Training Policy Extrapolation Yuxuan Liu 101 0 0 03 Mar 2025
The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agents Yihong Tang Kehai Chen X. Bai Zhengyu Niu Binghai Wang Jie Liu Min Zhang LLMAG 97 0 0 28 Feb 2025
Optimizing Large Language Models for ESG Activity Detection in Financial Texts Mattia Birti Francesco Osborne Andrea Maurino 86 0 0 28 Feb 2025
Conformal Linguistic Calibration: Trading-off between Factuality and Specificity Zhengping Jiang Anqi Liu Benjamin Van Durme 159 3 0 26 Feb 2025
Controlled Diversity: Length-optimized Natural Language Generation Diana Marie Schenke Timo Baumann 71 0 0 26 Feb 2025
ZEBRA: Leveraging Model-Behavioral Knowledge for Zero-Annotation Preference Dataset Construction Jeesu Jung Chanjun Park Sangkeun Jung 109 0 0 26 Feb 2025
Stackelberg Game Preference Optimization for Data-Efficient Alignment of Language Models Xu Chu Zhixin Zhang Tianyu Jia Yujie Jin 141 0 0 25 Feb 2025
CuDIP: Enhancing Theorem Proving in LLMs via Curriculum Learning-based Direct Preference Optimization Shuming Shi Ruobing Zuo Gaolei He Jianlin Wang Chenyang Xu Zhengfeng Yang 113 0 0 25 Feb 2025
AMPO: Active Multi-Preference Optimization for Self-play Preference Selection Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 104 0 0 25 Feb 2025
Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance Chenghua Huang Lu Wang Fangkai Yang Pu Zhao Zechao Li Qingwei Lin Dongmei Zhang Saravan Rajmohan Qi Zhang OffRL 80 1 0 24 Feb 2025
CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale Chenlong Wang Zhaoyang Chu Zhengxiang Cheng Xuyi Yang Kaiyue Qiu Yao Wan Zhou Zhao Xuanhua Shi Danny Chen ALM SyDa 101 0 0 23 Feb 2025
SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters Teng Xiao Yige Yuan Ziyang Chen Mingxiao Li Shangsong Liang Zhaochun Ren V. Honavar 260 11 0 21 Feb 2025
C3AI: Crafting and Evaluating Constitutions for Constitutional AI Yara Kyrychenko Ke Zhou Edyta Bogucka Daniele Quercia ELM 86 5 0 21 Feb 2025
SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation Song Duong Florian Le Bronnec Alexandre Allauzen Vincent Guigue Alberto Lumbreras Laure Soulier Patrick Gallinari HILM 102 0 0 20 Feb 2025
Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guarantees Yongtao Wu Luca Viano Yihang Chen Zhenyu Zhu Kimon Antonakopoulos Quanquan Gu Volkan Cevher 173 1 0 18 Feb 2025
Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models Yingqian Cui Pengfei He Jingying Zeng Hui Liu Xianfeng Tang ... Zhen Li Suhang Wang Yue Xing Jiliang Tang Qi He LRM 125 19 0 18 Feb 2025
LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data Cehao Yang Xueyuan Lin Chengjin Xu Xuhui Jiang Shengjie Ma Aofan Liu Hui Xiong Jian Guo LRM 59 2 0 18 Feb 2025
Personalized Top-k Set Queries Over Predicted Scores Sohrab Namazi Nia Subhodeep Ghosh Senjuti Basu Roy S. Amer-Yahia 154 0 0 18 Feb 2025
RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation C. Zhou Xinyu Zhang Dandan Song Xiancai Chen Wanli Gu Huipeng Ma Yuhang Tian Hao Fei Linmei Hu 98 2 0 13 Feb 2025
ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates L. Yang Zhaochen Yu Tengjiao Wang Mengdi Wang ReLM LRM AI4CE 173 18 0 10 Feb 2025
QExplorer: Large Language Model Based Query Extraction for Toxic Content Exploration Shaola Ren Li Ke Longtao Huang Dehong Gao Hui Xue 68 0 0 06 Feb 2025
COS(M+O)S: Curiosity and RL-Enhanced MCTS for Exploring Story Space via Language Models Tobias Materzok LRM 145 0 0 28 Jan 2025
Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking Benjamin Feuer Micah Goldblum Teresa Datta Sanjana Nambiar Raz Besaleli Samuel Dooley Max Cembalest John P. Dickerson ALM 153 0 0 28 Jan 2025
HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor Zihui Wu Haichang Gao Jiacheng Luo Zhaoxiang Liu 148 0 0 23 Jan 2025
In-situ graph reasoning and knowledge expansion using Graph-PReFLexOR Markus J. Buehler 48 0 0 14 Jan 2025
FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings Tong Liu Xiao Yu Wenxuan Zhou Jindong Gu Volker Tresp 75 1 0 11 Jan 2025
Verbosity-Aware Rationale Reduction: Effective Reduction of Redundant Rationale via Principled Criteria Joonwon Jang Jaehee Kim Wonbin Kweon Seonghyeon Lee Hwanjo Yu LRM 166 2 0 30 Dec 2024
REFA: Reference Free Alignment for multi-preference optimization Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 179 1 0 20 Dec 2024
Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model Yuzhong Hong Hanshan Zhang Junwei Bao Hongfei Jiang Yang Song OffRL 117 4 0 18 Dec 2024
MPPO: Multi Pair-wise Preference Optimization for LLMs with Arbitrary Negative Samples Shuo Xie Fangzhi Zhu Jiahui Wang Lulu Wen Wei Dai Xiaowei Chen Junxiong Zhu Kai Zhou Bo Zheng 86 0 0 13 Dec 2024
Learning to Reason via Self-Iterative Process Feedback for Small Language Models Kaiyuan Chen Jin Wang Xuejie Zhang LRM ReLM 110 2 0 11 Dec 2024
Classifier-free guidance in LLMs Safety Roman Smirnov MU 90 1 0 08 Dec 2024
FANAL -- Financial Activity News Alerting Language Modeling Framework Urjitkumar Patel Fang-Chun Yeh Chinmay Gondhalekar Hari Nalluri AIFin 113 0 0 04 Dec 2024
DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs Zhihan Liu Shenao Zhang Yongfei Liu Boyi Liu Yingxiang Yang Zhaoran Wang 179 5 0 20 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 134 93 1 15 Nov 2024
1-800-SHARED-TASKS @ NLU of Devanagari Script Languages: Detection of Language, Hate Speech, and Targets using LLMs Jebish Purbey Siddartha Pullakhandam Kanwal Mehreen Muhammad Arham Drishti Sharma Ashay Srivastava Ram Mohan Rao Kadiyala 59 0 0 11 Nov 2024
Combining Domain and Alignment Vectors to Achieve Better Knowledge-Safety Trade-offs in LLMs Megh Thakkar Yash More Quentin Fournier Matthew D Riemer Pin-Yu Chen Payel Das Payel Das MoMe 71 6 0 11 Nov 2024
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings Miguel Moura Ramos Tomás Almeida Daniel Vareta Filipe Azevedo Sweta Agrawal Patrick Fernandes André F. T. Martins 118 4 0 08 Nov 2024
GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation Models Nizar Islah Justine Gehring Diganta Misra Eilif B. Muller Irina Rish Terry Yue Zhuo Massimo Caccia SyDa 54 1 0 05 Nov 2024
PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment Dongxu Liu Bing Xu Yinzhuo Chen Bufan Xu Wenpeng Lu Muyun Yang Tiejun Zhao MoE 57 1 0 02 Nov 2024
TODO: Enhancing LLM Alignment with Ternary Preferences Yuxiang Guo Lu Yin Bo Jiang Jiaqi Zhang 120 3 0 02 Nov 2024
VPO: Leveraging the Number of Votes in Preference Optimization Jae Hyeon Cho Minkyung Park Byung-Jun Lee 30 2 0 30 Oct 2024
$f$ -PO: Generalizing Preference Optimization with $f$ -divergence Minimization Jiaqi Han Mingjian Jiang Yuxuan Song J. Leskovec Stefano Ermon 121 6 0 29 Oct 2024
UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function Zhichao Wang Bin Bi Z. Zhu Xiangbo Mao Jun Wang Shiyu Wang CLL 79 1 0 28 Oct 2024
Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model Wenhong Zhu Zhiwei He Xiaofeng Wang Pengfei Liu Rui Wang OSLM 107 6 0 24 Oct 2024