Self-Rewarding Language Models

18 January 2024

Xian Li

Jason Weston

Papers citing "Self-Rewarding Language Models"

50 / 276 papers shown

Title
Reinforcing General Reasoning without Verifiers Xiangxin Zhou Zichen Liu Anya Sims Haonan Wang Tianyu Pang Chongxuan Li Liang Wang Min Lin C. Du OffRL LRM 25 0 0 27 May 2025
Learning to Reason without External Rewards Xuandong Zhao Zhewei Kang Aosong Feng Sergey Levine Dawn Song OffRL ReLM LRM 32 2 0 26 May 2025
MR. Judge: Multimodal Reasoner as a Judge Renjie Pi Felix Bai Qibin Chen Simon Wang Jiulong Shan Kieran Liu Meng Cao ELM LRM 70 0 0 19 May 2025
Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space Hengli Li Chenxi Li Tong Wu Xuekai Zhu Yuxuan Wang ... Eric Hanchen Jiang Song-Chun Zhu Zixia Jia Ying Nian Wu Zilong Zheng LRM 70 0 0 19 May 2025
Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards Xiaoyuan Liu Tian Liang Zhiwei He Jiahao Xu Wenxuan Wang Pinjia He Zhaopeng Tu Haitao Mi Dong Yu OffRL ReLM LRM 64 0 0 19 May 2025
T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback Zehan Wang Ke Lei Chen Zhu Jiawei Huang Sashuai Zhou ... Xize Cheng Shengpeng Ji Zhenhui Ye Tao Jin Zhou Zhao 44 0 0 15 May 2025
Self Rewarding Self Improving Toby Simonds Kevin Lopez Akira Yoshiyama Dominique Garmier ReLM ALM LRM 65 0 0 12 May 2025
References Indeed Matter? Reference-Free Preference Optimization for Conversational Query Reformulation Doyoung Kim Youngjun Lee Joeun Kim Jihwan Bang Hwanjun Song Susik Yoon Jae-Gil Lee 113 0 0 10 May 2025
G-FOCUS: Towards a Robust Method for Assessing UI Design Persuasiveness Jaehyun Jeon Janghan Yoon Minsoo Kim Sumin Shim Yejin Choi Hanbin Kim Youngjae Yu AAML 74 0 0 08 May 2025
SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning Tianjian Li Daniel Khashabi 74 0 0 05 May 2025
A Survey on Progress in LLM Alignment from the Perspective of Reward Design Miaomiao Ji Yanqiu Wu Zhibin Wu Shoujin Wang Jian Yang Mark Dras Usman Naseem 54 2 0 05 May 2025
DeepCritic: Deliberate Critique with Large Language Models Wenkai Yang Jingwen Chen Yankai Lin Ji-Rong Wen ALM LRM 72 0 0 01 May 2025
Anyprefer: An Agentic Framework for Preference Data Synthesis Yiyang Zhou Zhaoxiang Wang Tianle Wang Shangyu Xing Peng Xia ... Chetan Bansal Weitong Zhang Ying Wei Joey Tianyi Zhou Huaxiu Yao 84 1 0 27 Apr 2025
Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors Ren-Wei Liang Chin-Ting Hsu Chan-Hung Yu Saransh Agrawal Shih-Cheng Huang Shang-Tse Chen Kuan-Hao Huang Shao-Hua Sun 106 0 0 27 Apr 2025
TTRL: Test-Time Reinforcement Learning Yuxin Zuo Kaiyan Zhang Li Sheng Li Sheng Xuekai Zhu ... Youbang Sun Zhiyuan Ma Lifan Yuan Ning Ding Bowen Zhou OffRL 275 17 0 22 Apr 2025
EvalAgent: Discovering Implicit Evaluation Criteria from the Web Manya Wadhwa Zayne Sprague Chaitanya Malaviya Philippe Laban Junyi Jessy Li Greg Durrett 83 1 0 21 Apr 2025
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators Yilun Zhou Austin Xu Peifeng Wang Caiming Xiong Shafiq Joty ELM ALM LRM 85 3 0 21 Apr 2025
Efficient MAP Estimation of LLM Judgment Performance with Prior Transfer Huaizhi Qu Inyoung Choi Zhen Tan Song Wang Sukwon Yun Qi Long Faizan Siddiqui Kwonjoon Lee Tianlong Chen 58 0 0 17 Apr 2025
A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future Jialun Zhong Wei Shen Yanzeng Li Songyang Gao Hua Lu Yicheng Chen Yang Zhang Wei Zhou Jinjie Gu Lei Zou LRM 69 8 0 12 Apr 2025
SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills Boyuan Zheng Michael Y. Fatemi Xiaolong Jin Ziyi Wang Apurva Gandhi ... Yu Gu Jayanth Srinivasa Gaowen Liu Graham Neubig Yu Su CLL 76 4 0 09 Apr 2025
EduPlanner: LLM-Based Multi-Agent Systems for Customized and Intelligent Instructional Design Xinsong Zhang Chao Zhang Jianwen Sun Jun Xiao Yi Yang Yawei Luo LLMAG AI4Ed 64 2 0 07 Apr 2025
MegaMath: Pushing the Limits of Open Math Corpora Fan Zhou Zengzhi Wang Nikhil Ranjan Zhoujun Cheng Liping Tang Guowei He Zhengzhong Liu Eric P. Xing LRM 86 2 0 03 Apr 2025
Inference-Time Scaling for Generalist Reward Modeling Zijun Liu P. Wang Ran Xu Shirong Ma Chong Ruan Ziwei Sun Yang Liu Y. Wu OffRL LRM 81 30 0 03 Apr 2025
CONGRAD:Conflicting Gradient Filtering for Multilingual Preference Alignment Jiangnan Li Thuy-Trang Vu Christian Herold Amirhossein Tebbifakhr Shahram Khadivi Gholamreza Haffari 75 0 0 31 Mar 2025
Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs Sanjoy Chowdhury Hanan Gani Nishit Anand Sayan Nag Ruohan Gao Mohamed Elhoseiny Salman Khan Dinesh Manocha LRM 115 0 0 29 Mar 2025
Preference-based Learning with Retrieval Augmented Generation for Conversational Question Answering Magdalena Kaiser Gerhard Weikum 71 1 0 28 Mar 2025
From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-reward Alignment Yucheng Suo Fan Ma Linchao Zhu T. Wang Fengyun Rao Yi Yang LRM 102 0 0 26 Mar 2025
reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs Zhaofeng Wu Michihiro Yasunaga Andrew Cohen Yoon Kim Asli Celikyilmaz Marjan Ghazvininejad 58 2 0 14 Mar 2025
RePO: ReLU-based Preference Optimization Junkang Wu Kexin Huang Xue Wang Jinyang Gao Bolin Ding Jiancan Wu Xiangnan He Xiang Wang 83 1 0 10 Mar 2025
SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting Linqi Yang Xiongwei Zhao Qihao Sun Ke Wang Ao Chen Peng Kang 3DGS 98 3 0 07 Mar 2025
SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winner Kejia Chen Jiawen Zhang Jiacong Hu Jiazhen Yang Jian Lou Zunlei Feng Mingli Song 98 0 0 06 Mar 2025
Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMs Yuzhe Gu Wentao Zhang Chengqi Lyu Dahua Lin Kai Chen 84 1 0 04 Mar 2025
Towards Effective and Efficient Context-aware Nucleus Detection in Histopathology Whole Slide Images Zhongyi Shui Ruizhe Guo Honglin Li Yuxuan Sun Yunlong Zhang Chenglu Zhu Jiatong Cai Pingyi Chen Yanzhou Su Lin Yang 63 2 0 04 Mar 2025
Can RLHF be More Efficient with Imperfect Reward Models? A Policy Coverage Perspective Jiawei Huang Bingcong Li Christoph Dann Niao He OffRL 156 1 0 26 Feb 2025
Self-rewarding correction for mathematical reasoning Wei Xiong Hanning Zhang Chenlu Ye Lichang Chen Nan Jiang Tong Zhang ReLM KELM LRM 103 13 0 26 Feb 2025
FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users Anikait Singh Sheryl Hsu Kyle Hsu E. Mitchell Stefano Ermon Tatsunori Hashimoto Archit Sharma Chelsea Finn SyDa OffRL 80 2 0 26 Feb 2025
Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data Siqi Guo Ilgee Hong Vicente Balmaseda Changlong Yu Liang Qiu Xin Liu Haoming Jiang Tuo Zhao Tianbao Yang 77 0 0 25 Feb 2025
Stackelberg Game Preference Optimization for Data-Efficient Alignment of Language Models Xu Chu Zhixin Zhang Tianyu Jia Yujie Jin 94 0 0 25 Feb 2025
Larger or Smaller Reward Margins to Select Preferences for Alignment? Kexin Huang Junkang Wu Ziqian Chen Xue Wang Jinyang Gao Bolin Ding Jiancan Wu Xiangnan He Xiang Wang 58 0 0 25 Feb 2025
Improving LLM General Preference Alignment via Optimistic Online Mirror Descent Yuheng Zhang Dian Yu Tao Ge Linfeng Song Zhichen Zeng Haitao Mi Nan Jiang Dong Yu 92 4 0 24 Feb 2025
RLTHF: Targeted Human Feedback for LLM Alignment Yifei Xu Tusher Chakraborty Emre Kıcıman Bibek Aryal Eduardo Rodrigues ... Rafael Padilha Leonardo Nunes Shobana Balakrishnan Songwu Lu Ranveer Chandra 135 1 0 24 Feb 2025
NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions Weizhe Yuan Jane Dwivedi-Yu Song Jiang Karthik Padthe Yang Li ... Ilia Kulikov Kyunghyun Cho Yuandong Tian Jason Weston Xian Li ReLM LRM 80 17 0 24 Feb 2025
Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale Fan Zhou Zengzhi Wang Qian Liu Junlong Li Pengfei Liu ALM 146 15 0 17 Feb 2025
Scaling Autonomous Agents via Automatic Reward Modeling And Planning Zhenfang Chen Delin Chen Rui Sun Wenjun Liu Chuang Gan LLMAG 76 4 0 17 Feb 2025
Preference Optimization for Reasoning with Pseudo Feedback Fangkai Jiao Geyang Guo Xingxing Zhang Nancy F. Chen Shafiq Joty Furu Wei LRM 153 12 0 17 Feb 2025
Preference learning made easy: Everything should be understood through win rate Lily H. Zhang Rajesh Ranganath 98 0 0 14 Feb 2025
Escaping Collapse: The Strength of Weak Data for Large Language Model Training Kareem Amin Sara Babakniya Alex Bie Weiwei Kong Umar Syed Sergei Vassilvitskii 85 3 0 13 Feb 2025
Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models Xin Zhou Yiwen Guo Ruotian Ma Tao Gui Qi Zhang Xuanjing Huang LRM 126 2 0 13 Feb 2025
STAIR: Improving Safety Alignment with Introspective Reasoning Yuanhang Zhang Siyuan Zhang Yao Huang Zeyu Xia Zhengwei Fang Xiao Yang Ranjie Duan Dong Yan Yinpeng Dong Jun Zhu LRM LLMSV 70 6 0 04 Feb 2025
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges Nayoung Lee Ziyang Cai Avi Schwarzschild Kangwook Lee Dimitris Papailiopoulos ReLM VLM LRM AI4CE 102 6 0 03 Feb 2025