On the Robustness of Reward Models for Language Model Alignment

On the Robustness of Reward Models for Language Model Alignment

12 May 2025

ArXiv (abs)PDF HTML

Papers citing "On the Robustness of Reward Models for Language Model Alignment"

17 / 17 papers shown

Title
Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs Chris Yuhao Liu Liang Zeng Qingbin Liu Rui Yan Jujie He Chaojie Wang Shuicheng Yan Yang Liu Yahui Zhou AI4TS 105 105 0 24 Oct 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 122 5 0 23 Oct 2024
RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style Yantao Liu Zijun Yao Rui Min Yixin Cao Lei Hou Juanzi Li OffRL ALM 91 38 0 21 Oct 2024
RMB: Comprehensively Benchmarking Reward Models in LLM Alignment Enyu Zhou Guodong Zheng Binghai Wang Zhiheng Xi Shihan Dou ... Yurong Mou Rui Zheng Tao Gui Qi Zhang Xuanjing Huang ALM 124 19 0 13 Oct 2024
RRM: Robust Reward Model Training Mitigates Reward Hacking Tianqi Liu Wei Xiong Jie Jessie Ren Lichang Chen Junru Wu ... Yuan Liu Bilal Piot Abe Ittycheriah Aviral Kumar Mohammad Saleh AAML 85 23 0 20 Sep 2024
From Lists to Emojis: How Format Bias Affects Model Alignment Xuanchang Zhang Wei Xiong Lichang Chen Dinesh Manocha Heng Huang Tong Zhang ALM 92 14 0 18 Sep 2024
Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer Zhihan Liu Miao Lu Shenao Zhang Boyi Liu Hongyi Guo Yingxiang Yang Jose H. Blanchet Zhaoran Wang 113 58 0 26 May 2024
OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework Jian Hu Xibin Wu Weixun Wang OpenLLMAI Team Dehao Zhang Yu Cao AI4CE VLM 90 130 0 20 May 2024
Advancing LLM Reasoning Generalists with Preference Trees Lifan Yuan Ganqu Cui Hanbin Wang Ning Ding Xingyao Wang ... Zhenghao Liu Bowen Zhou Hao Peng Zhiyuan Liu Maosong Sun LRM 122 119 0 02 Apr 2024
ORPO: Monolithic Preference Optimization without Reference Model Jiwoo Hong Noah Lee James Thorne OSLM 83 262 0 12 Mar 2024
Reward Model Ensembles Help Mitigate Overoptimization Thomas Coste Usman Anwar Robert Kirk David M. Krueger NoLa ALM 77 135 0 04 Oct 2023
LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion Dongfu Jiang Xiang Ren Bill Yuchen Lin ELM 78 320 0 05 Jun 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 387 4,125 0 29 May 2023
Mitigating Neural Network Overconfidence with Logit Normalization Hongxin Wei Renchunzi Xie Hao-Ran Cheng Lei Feng Bo An Yixuan Li OODD 220 285 0 19 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 880 13,148 0 04 Mar 2022
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models Samyam Rajbhandari Jeff Rasley Olatunji Ruwase Yuxiong He ALM AI4CE 82 902 0 04 Oct 2019
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 469 1,762 0 18 Sep 2019