Enhancing LLM Safety via Constrained Direct Preference Optimization

4 March 2024

Papers citing "Enhancing LLM Safety via Constrained Direct Preference Optimization"

17 / 17 papers shown

Title
Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning Nirav Diwan Tolga Ergen Dongsub Shim Honglak Lee 84 0 0 28 Jan 2025
Nash Learning from Human Feedback Rémi Munos Michal Valko Daniele Calandriello M. G. Azar Mark Rowland ... Nikola Momchev Olivier Bachem D. Mankowitz Doina Precup Bilal Piot 118 147 0 01 Dec 2023
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 180 647 0 18 Oct 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 295 1,518 0 27 Jul 2023
Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation Andi Peng Aviv Netanyahu Mark K. Ho Tianmin Shu Andreea Bobu J. Shah Pulkit Agrawal 60 14 0 12 Jul 2023
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models Wei Ping Weixin Chen Hengzhi Pei Chulin Xie Mintong Kang ... Zinan Lin Yuk-Kit Cheng Sanmi Koyejo Basel Alomair Yue Liu 124 431 0 20 Jun 2023
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards Alexandre Ramé Guillaume Couairon Mustafa Shukor Corentin Dancette Jean-Baptiste Gaya Laure Soulier Matthieu Cord MoMe 103 157 0 07 Jun 2023
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Zeqiu Wu Yushi Hu Weijia Shi Nouha Dziri Alane Suhr Prithviraj Ammanabrolu Noah A. Smith Mari Ostendorf Hannaneh Hajishirzi ALM 153 335 0 02 Jun 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 389 4,169 0 29 May 2023
Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Risk Chengyang Ying Xinning Zhou Hang Su Dong Yan Ning Chen Jun Zhu 64 44 0 09 Jun 2022
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 502 10,526 0 17 Jun 2021
Learning to summarize from human feedback Nisan Stiennon Long Ouyang Jeff Wu Daniel M. Ziegler Ryan J. Lowe Chelsea Voss Alec Radford Dario Amodei Paul Christiano ALM 262 2,192 0 02 Sep 2020
Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences Erdem Biyik Dylan P. Losey Malayandi Palan Nicholas C. Landolfi Gleb Shevchuk Dorsa Sadigh 75 118 0 24 Jun 2020
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 491 1,770 0 18 Sep 2019
Lyapunov-based Safe Policy Optimization for Continuous Control Yinlam Chow Ofir Nachum Aleksandra Faust Edgar A. Duénez-Guzmán Mohammad Ghavamzadeh 89 247 0 28 Jan 2019
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 577 19,315 0 20 Jul 2017
Deep reinforcement learning from human preferences Paul Christiano Jan Leike Tom B. Brown Miljan Martic Shane Legg Dario Amodei 218 3,377 0 12 Jun 2017