Learning to summarize from human feedback

2 September 2020

Papers citing "Learning to summarize from human feedback"

50 / 1,443 papers shown

Title
Stepwise Alignment for Constrained Language Model Policy Optimization Akifumi Wachi Thien Q. Tran Rei Sato Takumi Tanabe Yohei Akimoto 44 5 0 17 Apr 2024
Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study Shusheng Xu Wei Fu Jiaxuan Gao Wenjie Ye Weiling Liu Zhiyu Mei Guangju Wang Chao Yu Yi Wu 59 139 0 16 Apr 2024
Enhancing Confidence Expression in Large Language Models Through Learning from Past Experience Haixia Han Tingyun Li Shisong Chen Jie Shi Chengyu Du Yanghua Xiao Jiaqing Liang Xin Lin 53 6 0 16 Apr 2024
Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback Vincent Conitzer Rachel Freedman J. Heitzig Wesley H. Holliday Bob M. Jacobs ... Eric Pacuit Stuart Russell Hailey Schoelkopf Emanuel Tewolde W. Zwicker 53 30 0 16 Apr 2024
Reinforcement Learning from Multi-role Debates as Feedback for Bias Mitigation in LLMs Ruoxi Cheng Haoxuan Ma Shuirong Cao Jiaqi Li Aihua Pei Zhiqiang Wang Pengliang Ji Haoyu Wang Jiaqi Huo AI4CE 39 6 0 15 Apr 2024
LLM Evaluators Recognize and Favor Their Own Generations Arjun Panickssery Samuel R. Bowman Shi Feng 60 159 0 15 Apr 2024
Impact of Preference Noise on the Alignment Performance of Generative Language Models Yang Gao Dana Alon Donald Metzler 45 17 0 15 Apr 2024
Learn Your Reference Model for Real Good Alignment Alexey Gorbatovski Boris Shaposhnikov Alexey Malakhov Nikita Surnachev Yaroslav Aksenov Ian Maksimov Nikita Balagansky Daniil Gavrilov OffRL 61 28 0 15 Apr 2024
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs Shreyas Chaudhari Pranjal Aggarwal Vishvak Murahari Tanmay Rajpurohit Ashwin Kalyan Karthik Narasimhan Ameet Deshpande Bruno Castro da Silva 36 35 0 12 Apr 2024
Dataset Reset Policy Optimization for RLHF Jonathan D. Chang Wenhao Zhan Owen Oertell Kianté Brantley Dipendra Kumar Misra Jason D. Lee Wen Sun OffRL 32 21 0 12 Apr 2024
High-Dimension Human Value Representation in Large Language Models Samuel Cahyawijaya Delong Chen Yejin Bang Leila Khalatbari Bryan Wilie Ziwei Ji Etsuko Ishii Pascale Fung 71 5 0 11 Apr 2024
"We Need Structured Output": Towards User-centered Constraints on Large Language Model Output Michael Xieyang Liu Frederick Liu Alexander J. Fiannaca Terry Koo Lucas Dixon Michael Terry Carrie J. Cai 51 25 0 10 Apr 2024
Improving Language Model Reasoning with Self-motivated Learning Yunlong Feng Yang Xu Libo Qin Yasheng Wang Wanxiang Che LRM ReLM 42 7 0 10 Apr 2024
Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning Ruiqi Zhang Licong Lin Yu Bai Song Mei MU 74 139 0 08 Apr 2024
Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Data Tim Baumgärtner Yang Gao Dana Alon Donald Metzler AAML 41 18 0 08 Apr 2024
Towards Understanding the Influence of Reward Margin on Preference Model Performance Bowen Qin Duanyu Feng Xi Yang 42 3 0 07 Apr 2024
Binary Classifier Optimization for Large Language Model Alignment Seungjae Jung Gunsoo Han D. W. Nam Kyoung-Woon On 42 21 0 06 Apr 2024
Aligning Diffusion Models by Optimizing Human Utility Shufan Li Konstantinos Kallidromitis Akash Gokul Yusuke Kato Kazuki Kozuka 109 30 0 06 Apr 2024
ROPO: Robust Preference Optimization for Large Language Models Xize Liang Chao Chen Shuang Qiu Jie Wang Yue-bo Wu Zhihang Fu Zhihao Shi Feng Wu Jieping Ye 53 2 0 05 Apr 2024
Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data Jingyu Zhang Marc Marone Tianjian Li Benjamin Van Durme Daniel Khashabi 93 9 0 05 Apr 2024
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences Corby Rosset Ching-An Cheng Arindam Mitra Michael Santacroce Ahmed Hassan Awadallah Tengyang Xie 152 116 0 04 Apr 2024
Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought Jooyoung Lee Fan Yang Thanh Tran Qian Hu Emre Barut Kai-Wei Chang Chengwei Su ReLM LLMAG LRM 21 10 0 04 Apr 2024
HyperCLOVA X Technical Report Kang Min Yoo Jaegeun Han Sookyo In Heewon Jeon Jisu Jeong ... Hyunkyung Noh Se-Eun Choi Sang-Woo Lee Jung Hwa Lim Nako Sung VLM 44 8 0 02 Apr 2024
Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models Kyuyoung Kim Jongheon Jeong Minyong An Mohammad Ghavamzadeh Krishnamurthy Dvijotham Jinwoo Shin Kimin Lee EGVM 42 6 0 02 Apr 2024
Asymptotics of Language Model Alignment Joy Qiping Yang Salman Salamatian Ziteng Sun A. Suresh Ahmad Beirami 69 23 0 02 Apr 2024
Conjugate-Gradient-like Based Adaptive Moment Estimation Optimization Algorithm for Deep Learning Jiawu Tian Liwei Xu Xiaowei Zhang Yongqi Li ODL 56 0 0 02 Apr 2024
CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models Xuechen Liang Meiling Tao Yinghui Xia Yiting Xie Jun Wang JingSong Yang LLMAG 35 12 0 02 Apr 2024
Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignment Yuu Jinnai Tetsuro Morimura Kaito Ariu Kenshi Abe 71 7 0 01 Apr 2024
Prior Constraints-based Reward Model Training for Aligning Large Language Models Hang Zhou Chenglong Wang Yimin Hu Tong Xiao Chunliang Zhang Jingbo Zhu ALM 51 2 0 01 Apr 2024
A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias Yuemei Xu Ling Hu Jiayi Zhao Zihan Qiu Yuqi Ye Hanwen Gu LRM 37 37 0 01 Apr 2024
Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization Hritik Bansal Ashima Suvarna Gantavya Bhatt Nanyun Peng Kai-Wei Chang Aditya Grover ALM 64 9 0 31 Mar 2024
Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods Yuji Cao Huan Zhao Yuheng Cheng Ting Shu Guolong Liu Gaoqi Liang Junhua Zhao Yun Li LLMAG KELM OffRL LM&Ro 48 51 0 30 Mar 2024
A Survey of using Large Language Models for Generating Infrastructure as Code Kalahasti Ganesh Srivatsa Sabyasachi Mukhopadhyay Ganesh Katrapati Manish Shrivastava 41 1 0 30 Mar 2024
MANGO: A Benchmark for Evaluating Mapping and Navigation Abilities of Large Language Models Peng Ding Jiading Fang Peng Li Kangrui Wang Xiaochen Zhou Mo Yu Jing Li Matthew R. Walter Hongyuan Mei RALM ELM 53 6 0 29 Mar 2024
Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model Qi Gou Cam-Tu Nguyen 35 8 0 28 Mar 2024
Fine-Tuning Language Models with Reward Learning on Policy Hao Lang Fei Huang Yongbin Li ALM 42 5 0 28 Mar 2024
Disentangling Length from Quality in Direct Preference Optimization Ryan Park Rafael Rafailov Stefano Ermon Chelsea Finn ALM 56 112 0 28 Mar 2024
CYCLE: Learning to Self-Refine the Code Generation Yangruibo Ding Marcus J. Min Gail E. Kaiser Baishakhi Ray 41 29 0 27 Mar 2024
Understanding the Learning Dynamics of Alignment with Human Feedback Shawn Im Yixuan Li ALM 37 11 0 27 Mar 2024
Improving Attributed Text Generation of Large Language Models via Preference Learning Dongfang Li Zetian Sun Baotian Hu Zhenyu Liu Xinshuo Hu Xuebo Liu Min Zhang 53 13 0 27 Mar 2024
Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback Hongshen Xu Zichen Zhu Situo Zhang Da Ma Shuai Fan Lu Chen Kai Yu HILM 41 35 0 27 Mar 2024
Assessment of Multimodal Large Language Models in Alignment with Human Values Zhelun Shi Zhipin Wang Hongxing Fan Zaibin Zhang Lijun Li Yongting Zhang Zhen-fei Yin Lu Sheng Yu Qiao Jing Shao 47 16 0 26 Mar 2024
MetaAligner: Towards Generalizable Multi-Objective Alignment of Language Models Kailai Yang Zhiwei Liu Qianqian Xie Jimin Huang Tianlin Zhang Sophia Ananiadou 37 15 0 25 Mar 2024
If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions Reza Esfandiarpoor Cristina Menghini Stephen H. Bach CoGe VLM 45 8 0 25 Mar 2024
The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization Shengyi Huang Michael Noukhovitch Arian Hosseini Kashif Rasul Weixun Wang Lewis Tunstall VLM 30 31 0 24 Mar 2024
Risk and Response in Large Language Models: Evaluating Key Threat Categories Bahareh Harandizadeh A. Salinas Fred Morstatter 30 3 0 22 Mar 2024
DreamReward: Text-to-3D Generation with Human Preference Junliang Ye Fangfu Liu Qixiu Li Zhengyi Wang Yikai Wang Xinzhou Wang Yueqi Duan Jun Zhu 74 21 0 21 Mar 2024
Reinforcement Learning from Reflective Feedback (RLRF): Aligning and Improving LLMs via Fine-Grained Self-Reflection Kyungjae Lee Dasol Hwang Sunghyun Park Youngsoo Jang Moontae Lee 46 8 0 21 Mar 2024
Chain-of-Interaction: Enhancing Large Language Models for Psychiatric Behavior Understanding by Dyadic Contexts Guangzeng Han Weisi Liu Xiaolei Huang Brian Borsari 41 21 0 20 Mar 2024
Llama meets EU: Investigating the European Political Spectrum through the Lens of LLMs Ilias Chalkidis Stephanie Brandl 36 7 0 20 Mar 2024