Language Model Alignment with Elastic Reset

6 December 2023

Aaron Courville

Papers citing "Language Model Alignment with Elastic Reset"

26 / 26 papers shown

Title
Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance Chenghua Huang Lu Wang Fangkai Yang Pu Zhao ZeLin Li Qingwei Lin Dongmei Zhang Saravan Rajmohan Qi Zhang OffRL 57 1 0 24 Feb 2025
Learning to Summarize from LLM-generated Feedback Hwanjun Song Taewon Yun Yuho Lee Jihwan Oh Gihun Lee Jason (Jinglun) Cai Hang Su 73 2 0 28 Jan 2025
Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models Michael Noukhovitch Shengyi Huang Sophie Xhonneux Arian Hosseini Rishabh Agarwal Aaron C. Courville OffRL 79 5 0 23 Oct 2024
Understanding Likelihood Over-optimisation in Direct Alignment Algorithms Zhengyan Shi Sander Land Acyr Locatelli Matthieu Geist Max Bartolo 46 4 0 15 Oct 2024
Reward-Augmented Data Enhances Direct Preference Alignment of LLMs Shenao Zhang Zhihan Liu Boyi Liu Wenjie Qu Yingxiang Yang Y. Liu Liyu Chen Tao Sun Ziyi Wang 101 3 0 10 Oct 2024
Neuroplastic Expansion in Deep Reinforcement Learning Jiashun Liu J. Obando-Ceron Aaron C. Courville L. Pan 42 3 0 10 Oct 2024
SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection Han Shen Pin-Yu Chen Payel Das Tianyi Chen ALM 26 11 0 09 Oct 2024
Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learning Hao Ma Tianyi Hu Zhiqiang Pu Boyin Liu Xiaolin Ai Yanyan Liang Min Chen 50 3 0 08 Oct 2024
Diffusion Guided Language Modeling Justin Lovelace Varsha Kishore Yiwei Chen Kilian Q. Weinberger 44 6 0 08 Aug 2024
The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Models Zihui Wu Haichang Gao Jianping He Ping Wang 29 6 0 25 Jul 2024
Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing Huanqian Wang Yang Yue Rui Lu Jingxin Shi Andrew Zhao Shenzhi Wang Shiji Song Gao Huang LM&Ro KELM 51 6 0 11 Jul 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 59 14 0 24 Jun 2024
On the Transformations across Reward Model, Parameter Update, and In-Context Prompt Deng Cai Huayang Li Tingchen Fu Siheng Li Weiwen Xu ... Leyang Cui Yan Wang Lemao Liu Taro Watanabe Shuming Shi KELM 30 2 0 24 Jun 2024
Self-Exploring Language Models: Active Preference Elicitation for Online Alignment Shenao Zhang Donghan Yu Hiteshi Sharma Ziyi Yang Shuohang Wang Hany Hassan Zhaoran Wang LRM 48 28 0 29 May 2024
Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment Keming Lu Bowen Yu Fei Huang Yang Fan Runji Lin Chang Zhou MoMe 29 18 0 28 May 2024
Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learning Mathieu Rita Florian Strub Rahma Chaabouni Paul Michel Emmanuel Dupoux Olivier Pietquin 42 8 0 30 Apr 2024
URL: Universal Referential Knowledge Linking via Task-instructed Representation Compression Zhuoqun Li Hongyu Lin Tianshu Wang Boxi Cao Yaojie Lu Weixiang Zhou Hao Wang Zhenyu Zeng Le Sun Xianpei Han 48 1 0 24 Apr 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 57 22 0 22 Apr 2024
The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization Shengyi Huang Michael Noukhovitch Arian Hosseini Kashif Rasul Weixun Wang Lewis Tunstall VLM 30 31 0 24 Mar 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 116 93 0 22 Jan 2024
Mitigating the Alignment Tax of RLHF Yong Lin Hangyu Lin Wei Xiong Shizhe Diao Zeming Zheng ... Han Zhao Nan Jiang Heng Ji Yuan Yao Tong Zhang MoMe CLL 29 67 0 12 Sep 2023
Emergent Communication: Generalization and Overfitting in Lewis Games Mathieu Rita Corentin Tallec Paul Michel Jean-Bastien Grill Olivier Pietquin Emmanuel Dupoux Florian Strub AI4CE 90 23 0 30 Sep 2022
The Primacy Bias in Deep Reinforcement Learning Evgenii Nikishin Max Schwarzer P. DÓro Pierre-Luc Bacon Aaron C. Courville OnRL 96 180 0 16 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 325 11,953 0 04 Mar 2022
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 338 5,785 0 29 Apr 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 286 1,595 0 18 Sep 2019