Dense Reward for Free in Reinforcement Learning from Human Feedback

1 February 2024

Papers citing "Dense Reward for Free in Reinforcement Learning from Human Feedback"

29 / 29 papers shown

Title
Reward Inside the Model: A Lightweight Hidden-State Reward Model for LLM's Best-of-N sampling Jizhou Guo Zhaomin Wu Philip S. Yu 4 0 0 18 May 2025
Attention-Based Reward Shaping for Sparse and Delayed Rewards Ian Holmes Min Chi OffRL 29 0 0 16 May 2025
A Survey on Progress in LLM Alignment from the Perspective of Reward Design Miaomiao Ji Yanqiu Wu Zhibin Wu Shoujin Wang Jian Yang Mark Dras Usman Naseem 41 1 0 05 May 2025
Learning Explainable Dense Reward Shapes via Bayesian Optimization Ryan Koo Ian Yang Vipul Raheja Mingyi Hong Kwang-Sung Jun Dongyeop Kang 31 0 0 22 Apr 2025
Reciprocity-Aware Convolutional Neural Networks for Map-Based Path Loss Prediction Ryan Dempsey Jonathan Ethier Halim Yanikomeroglu 44 0 0 04 Apr 2025
MT-RewardTree: A Comprehensive Framework for Advancing LLM-Based Machine Translation via Reward Modeling Zhaopeng Feng Jiahan Ren Jiayuan Su Jiamei Zheng Zhihang Tang Hongwei Wang Zuozhu Liu LRM 65 1 0 15 Mar 2025
AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation Songming Zhang Xue Zhang Tong Zhang Bojie Hu Yufeng Chen Jinan Xu 52 1 0 04 Mar 2025
Advantage-Guided Distillation for Preference Alignment in Small Language Models Shiping Gao Fanqi Wan Jiajian Guo Xiaojun Quan Qifan Wang ALM 58 0 0 25 Feb 2025
Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data Siqi Guo Ilgee Hong Vicente Balmaseda Changlong Yu Liang Qiu Xin Liu Haoming Jiang Tuo Zhao Tianbao Yang 50 0 0 25 Feb 2025
Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning Hao Sun M. Schaar 94 14 0 28 Jan 2025
Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model Yueqin Yin Shentao Yang Yujia Xie Ziyi Yang Yuting Sun Hany Awadalla Weizhu Chen Mingyuan Zhou 52 1 0 07 Jan 2025
Interpreting Language Reward Models via Contrastive Explanations Junqi Jiang Tom Bewley Saumitra Mishra Freddy Lecue Manuela Veloso 76 1 0 25 Nov 2024
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings Miguel Moura Ramos Tomás Almeida Daniel Vareta Filipe Azevedo Sweta Agrawal Patrick Fernandes André F. T. Martins 33 1 0 08 Nov 2024
TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling Jiahao Qiu Yifu Lu Yifan Zeng Jiacheng Guo Jiayi Geng Huazheng Wang Kaixuan Huang Yue Wu Mengdi Wang 54 22 0 18 Oct 2024
Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategies Jiajie Yu Yuhong Wang Wei Ma OffRL 44 1 0 14 Oct 2024
Alignment of Diffusion Models: Fundamentals, Challenges, and Future Buhua Liu Shitong Shao Bao Li Lichen Bai Zhiqiang Xu Haoyi Xiong James Kwok Sumi Helal Zeke Xie 45 12 0 11 Sep 2024
Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback Jiayi Zhou Yalan Qin Juntao Dai Yaodong Yang 41 4 0 30 Aug 2024
Selective Preference Optimization via Token-Level Reward Function Estimation Kailai Yang Zhiwei Liu Qianqian Xie Jimin Huang Erxue Min Sophia Ananiadou 33 10 0 24 Aug 2024
Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization Yuxin Jiang Bo Huang Yufei Wang Xingshan Zeng Liangyou Li Yasheng Wang Xin Jiang Lifeng Shang Ruiming Tang Wei Wang 44 5 0 14 Aug 2024
TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedback Eunseop Yoon Hee Suk Yoon Soohwan Eom Gunsoo Han D. W. Nam DaeJin Jo Kyoung-Woon On M. Hasegawa-Johnson Sungwoong Kim C. Yoo ALM 38 15 0 23 Jul 2024
Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning Jifan Zhang Lalit P. Jain Yang Guo Jiayi Chen Kuan Lok Zhou ... Scott Sievert Timothy T. Rogers Kevin Jamieson Robert Mankoff Robert Nowak 39 5 0 15 Jun 2024
Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs Rui Yang Ruomeng Ding Yong Lin Huan Zhang Tong Zhang 44 43 0 14 Jun 2024
Discovering Preference Optimization Algorithms with and for Large Language Models Chris Xiaoxuan Lu Samuel Holt Claudio Fanconi Alex J. Chan Jakob Foerster M. Schaar R. T. Lange OffRL 40 16 0 12 Jun 2024
RLHF Workflow: From Reward Modeling to Online RLHF Hanze Dong Wei Xiong Bo Pang Haoxiang Wang Han Zhao Yingbo Zhou Nan Jiang Doyen Sahoo Caiming Xiong Tong Zhang OffRL 29 99 0 13 May 2024
DPO Meets PPO: Reinforced Token Optimization for RLHF Han Zhong Zikang Shan Guhao Feng Li Zhao Di He Jiang Bian Di He Jiang Bian Liwei Wang 57 57 0 29 Apr 2024
Stabilizing RLHF through Advantage Model and Selective Rehearsal Baolin Peng Linfeng Song Ye Tian Lifeng Jin Haitao Mi Dong Yu 37 17 0 18 Sep 2023
Learning Guidance Rewards with Trajectory-space Smoothing Tanmay Gangwani Yuanshuo Zhou Jian Peng 26 33 0 23 Oct 2020
Reinforcement Learning with Random Delays Simon Ramstedt Yann Bouteiller Giovanni Beltrame C. Pal Jonathan Binas 127 59 0 06 Oct 2020
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,748 0 26 Sep 2016