Stabilizing RLHF through Advantage Model and Selective Rehearsal

Stabilizing RLHF through Advantage Model and Selective Rehearsal

18 September 2023

Linfeng Song

Dong Yu

Papers citing "Stabilizing RLHF through Advantage Model and Selective Rehearsal"

14 / 14 papers shown

Title
Improving LLM General Preference Alignment via Optimistic Online Mirror Descent Yuheng Zhang Dian Yu Tao Ge Linfeng Song Zhichen Zeng Haitao Mi Nan Jiang Dong Yu 122 4 0 24 Feb 2025
Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates Hui Wei Shenghua He Tian Xia Andy H. Wong Jingyang Lin Mei Han Mei Han ALM ELM 124 31 0 23 Aug 2024
Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning Yuheng Zhang Dian Yu Baolin Peng Linfeng Song Ye Tian Mingyue Huo Nan Jiang Haitao Mi Dong Yu 186 18 0 30 Jun 2024
Reinforced Self-Training (ReST) for Language Modeling Çağlar Gülçehre T. Paine S. Srinivasan Ksenia Konyushkova L. Weerts ... Chenjie Gu Wolfgang Macherey Arnaud Doucet Orhan Firat Nando de Freitas OffRL 118 298 0 17 Aug 2023
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment Hanze Dong Wei Xiong Deepanshu Goyal Yihan Zhang Winnie Chow Boyao Wang Shizhe Diao Jipeng Zhang Kashun Shum Tong Zhang ALM 76 456 0 13 Apr 2023
RRHF: Rank Responses to Align Language Models with Human Feedback without tears Zheng Yuan Hongyi Yuan Chuanqi Tan Wei Wang Songfang Huang Feiran Huang ALM 159 374 0 11 Apr 2023
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 196 1,618 0 15 Dec 2022
ValueNet: A New Dataset for Human Value Driven Dialogue System Liang Qiu Yizhou Zhao Jinchao Li Pan Lu Baolin Peng Jianfeng Gao Song-Chun Zhu 64 36 0 12 Dec 2021
A General Language Assistant as a Laboratory for Alignment Amanda Askell Yuntao Bai Anna Chen Dawn Drain Deep Ganguli ... Tom B. Brown Jack Clark Sam McCandlish C. Olah Jared Kaplan ALM 118 779 0 01 Dec 2021
SimCSE: Simple Contrastive Learning of Sentence Embeddings Tianyu Gao Xingcheng Yao Danqi Chen AILaw SSL 261 3,396 0 18 Apr 2021
Towards Continual Reinforcement Learning: A Review and Perspectives Khimya Khetarpal Matthew D Riemer Irina Rish Doina Precup CLL OffRL 109 320 0 25 Dec 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 159 2,737 0 05 Jun 2020
Scalable agent alignment via reward modeling: a research direction Jan Leike David M. Krueger Tom Everitt Miljan Martic Vishal Maini Shane Legg 95 413 0 19 Nov 2018
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 499 19,065 0 20 Jul 2017