Deep reinforcement learning from human preferences

12 June 2017

Papers citing "Deep reinforcement learning from human preferences"

50 / 216 papers shown

Title
$f$ -PO: Generalizing Preference Optimization with $f$ -divergence Minimization Jiaqi Han Mingjian Jiang Yuxuan Song J. Leskovec Stefano Ermon 98 5 0 29 Oct 2024
L3Ms -- Lagrange Large Language Models Guneet S. Dhillon Xingjian Shi Yee Whye Teh Alex Smola 422 0 0 28 Oct 2024
Enhancing Safety in Reinforcement Learning with Human Feedback via Rectified Policy Optimization Xiyue Peng Hengquan Guo Jiawei Zhang Dongqing Zou Ziyu Shao Honghao Wei Xin Liu 92 2 0 25 Oct 2024
Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback Lester James V. Miranda Yizhong Wang Yanai Elazar Sachin Kumar Valentina Pyatkin Faeze Brahman Noah A. Smith Hannaneh Hajishirzi Pradeep Dasigi 96 12 0 24 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 135 5 0 24 Oct 2024
End-to-end Training for Recommendation with Language-based User Profiles Zhaolin Gao Joyce Zhou Yijia Dai Thorsten Joachims AI4Ed 120 4 0 24 Oct 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 103 5 0 23 Oct 2024
Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models Michael Noukhovitch Shengyi Huang Sophie Xhonneux Arian Hosseini Rishabh Agarwal Rameswar Panda OffRL 128 11 0 23 Oct 2024
Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment Mingzhi Wang Chengdong Ma Qizhi Chen Linjian Meng Yang Han Jiancong Xiao Zhaowei Zhang Jing Huo Weijie Su Yaodong Yang 126 9 0 22 Oct 2024
On The Global Convergence Of Online RLHF With Neural Parametrization Mudit Gaur Amrit Singh Bedi Raghu Pasupathy Vaneet Aggarwal 63 1 0 21 Oct 2024
M-RewardBench: Evaluating Reward Models in Multilingual Settings Srishti Gureja Lester James V. Miranda Shayekh Bin Islam Rishabh Maheshwary Drishti Sharma Gusti Winata Nathan Lambert Sebastian Ruder Sara Hooker Marzieh Fadaee LRM 97 21 0 20 Oct 2024
Contextual Augmented Multi-Model Programming (CAMP): A Hybrid Local-Cloud Copilot Framework Yuchen Wang Shangxin Guo C. Tan 58 0 0 20 Oct 2024
Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning H. Fernando Han Shen Parikshit Ram Yi Zhou Horst Samulowitz Nathalie Baracaldo Tianyi Chen CLL 114 4 0 20 Oct 2024
JudgeBench: A Benchmark for Evaluating LLM-based Judges Sijun Tan Siyuan Zhuang Kyle Montgomery William Y. Tang Alejandro Cuadron Chenguang Wang Raluca A. Popa Ion Stoica ELM ALM 115 49 0 16 Oct 2024
Preference Optimization with Multi-Sample Comparisons Chaoqi Wang Zhuokai Zhao Chen Zhu Karthik Abinav Sankararaman Michal Valko ... Zhaorun Chen Madian Khabsa Yuxin Chen Hao Ma Sinong Wang 108 10 0 16 Oct 2024
MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation S. Gorti Ilan Gofman Zhaoyan Liu Jiapeng Wu Noël Vouitsis Guangwei Yu Jesse C. Cresswell Rasa Hosseinzadeh SyDa 93 12 0 16 Oct 2024
DeformPAM: Data-Efficient Learning for Long-horizon Deformable Object Manipulation via Preference-based Action Alignment Wendi Chen Han Xue Fangyuan Zhou Yuan Fang Cewu Lu 82 1 0 15 Oct 2024
Offline Model-Based Optimization by Learning to Rank Rong-Xi Tan Ke Xue Shen-Huan Lyu Haopu Shang Yao Wang Yaoyuan Wang Sheng Fu Chao Qian OffRL 99 3 0 15 Oct 2024
Taming Overconfidence in LLMs: Reward Calibration in RLHF Jixuan Leng Chengsong Huang Banghua Zhu Jiaxin Huang 88 14 0 13 Oct 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 431 3 0 11 Oct 2024
Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization Guanlin Liu Kaixuan Ji Ning Dai Zheng Wu Chen Dun Q. Gu Lin Yan Quanquan Gu Lin Yan OffRL LRM 113 12 0 11 Oct 2024
TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees Weibin Liao Xu Chu Yasha Wang LRM 104 7 0 10 Oct 2024
COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act Philipp Guldimann Alexander Spiridonov Robin Staab Nikola Jovanović Mark Vero ... Mislav Balunović Nikola Konstantinov Pavol Bielik Petar Tsankov Martin Vechev ELM 85 7 0 10 Oct 2024
ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time Yi Ding Bolian Li Ruqi Zhang MLLM 104 13 0 09 Oct 2024
Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning Chongyu Fan Jiancheng Liu Licong Lin Jinghan Jia Ruiqi Zhang Song Mei Sijia Liu MU 142 30 0 09 Oct 2024
TLDR: Token-Level Detective Reward Model for Large Vision Language Models Deqing Fu Tong Xiao Rui Wang Wang Zhu Pengchuan Zhang Guan Pang Robin Jia Lawrence Chen 117 7 0 07 Oct 2024
Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF Zhaolin Gao Wenhao Zhan Jonathan D. Chang Gokul Swamy Kianté Brantley Jason D. Lee Wen Sun OffRL 130 7 0 06 Oct 2024
MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions Yekun Chai Haoran Sun Huang Fang Shuohuan Wang Yu Sun Hua Wu 425 3 0 03 Oct 2024
The Role of Deductive and Inductive Reasoning in Large Language Models Chengkun Cai Xu Zhao Haoliang Liu Zhongyu Jiang Tianfang Zhang Zongkai Wu Lei Li Lei Li Lei Li LRM 91 3 0 03 Oct 2024
Endless Jailbreaks with Bijection Learning Brian R. Y. Huang Maximilian Li Leonard Tang AAML 114 8 0 02 Oct 2024
HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models Seanie Lee Haebin Seong Dong Bok Lee Minki Kang Xiaoyin Chen Dominik Wagner Yoshua Bengio Juho Lee Sung Ju Hwang 164 6 0 02 Oct 2024
Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data Sreyan Ghosh Sonal Kumar Zhifeng Kong Rafael Valle Bryan Catanzaro Dinesh Manocha DiffM 102 3 0 02 Oct 2024
Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown Xingzhou Lou Dong Yan Wei Shen Yuzi Yan Jian Xie Junge Zhang 177 27 0 01 Oct 2024
SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning Rimvydas Rubavicius Peter David Fagan A. Lascarides Subramanian Ramamoorthy LM&Ro 396 0 0 26 Sep 2024
Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits Tuhin Chakrabarty Philippe Laban Chien-Sheng Wu 73 13 0 22 Sep 2024
Addressing and Visualizing Misalignments in Human Task-Solving Trajectories Sejin Kim Hosung Lee Sundong Kim 96 0 0 21 Sep 2024
Aligning Language Models Using Follow-up Likelihood as Reward Signal Chen Zhang Dading Chong Feng Jiang Chengguang Tang Anningzhe Gao Guohua Tang Haizhou Li ALM 71 2 0 20 Sep 2024
RRM: Robust Reward Model Training Mitigates Reward Hacking Tianqi Liu Wei Xiong Jie Jessie Ren Lichang Chen Junru Wu ... Yuan Liu Bilal Piot Abe Ittycheriah Aviral Kumar Mohammad Saleh AAML 78 21 0 20 Sep 2024
From Lists to Emojis: How Format Bias Affects Model Alignment Xuanchang Zhang Wei Xiong Lichang Chen Dinesh Manocha Heng Huang Tong Zhang ALM 87 13 0 18 Sep 2024
Your Weak LLM is Secretly a Strong Teacher for Alignment Leitian Tao Yixuan Li 120 8 0 13 Sep 2024
From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning Wei Chen Zhen Huang Liang Xie Binbin Lin Houqiang Li ... Deng Cai Yonggang Zhang Wenxiao Wang Xu Shen Jieping Ye 105 9 0 03 Sep 2024
Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Wenxuan Zhang Philip Torr Mohamed Elhoseiny Adel Bibi 152 14 0 27 Aug 2024
Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization Yuxin Jiang Bo Huang Yufei Wang Xingshan Zeng Liangyou Li Yasheng Wang Xin Jiang Lifeng Shang Ruiming Tang Wei Wang 85 7 0 14 Aug 2024
Problem Solving Through Human-AI Preference-Based Cooperation Subhabrata Dutta Timo Kaufmann Goran Glavaš Ivan Habernal Kristian Kersting Frauke Kreuter Mira Mezini Iryna Gurevych Eyke Hüllermeier Hinrich Schuetze 184 1 0 14 Aug 2024
Natural Language Outlines for Code: Literate Programming in the LLM Era Kensen Shi Deniz Altınbüken Saswat Anand Mihai Christodorescu Katja Grünwedel ... Tobias Welp Pengcheng Yin Manzil Zaheer Satish Chandra Charles Sutton 92 6 0 09 Aug 2024
Tamper-Resistant Safeguards for Open-Weight LLMs Rishub Tamirisa Bhrugu Bharathi Long Phan Andy Zhou Alice Gatti ... Andy Zou Dawn Song Bo Li Dan Hendrycks Mantas Mazeika AAML MU 94 56 0 01 Aug 2024
Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift Seongho Son William Bankes Sayak Ray Chowdhury Brooks Paige Ilija Bogunovic 81 4 0 26 Jul 2024
PECAN: Personalizing Robot Behaviors through a Learned Canonical Space Heramb Nemlekar Robert Ramirez Sanchez Dylan P. Losey 91 3 0 22 Jul 2024
A Unified Confidence Sequence for Generalized Linear Models, with Applications to Bandits Junghyun Lee Se-Young Yun Kwang-Sung Jun 141 6 0 19 Jul 2024
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training Youliang Yuan Wenxiang Jiao Wenxuan Wang Jen-tse Huang Jiahao Xu Tian Liang Pinjia He Zhaopeng Tu 88 29 0 12 Jul 2024