BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset

10 July 2023

Jiaming Ji

Juntao Dai

Chi Zhang

Chi Zhang

Papers citing "BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset"

50 / 97 papers shown

Title
Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety Zihan Guan Mengxuan Hu Ronghang Zhu Sheng Li Anil Vullikanti AAML 31 0 0 11 May 2025
DMRL: Data- and Model-aware Reward Learning for Data Extraction Zhiqiang Wang Ruoxi Cheng 31 0 0 07 May 2025
Sailing AI by the Stars: A Survey of Learning from Rewards in Post-Training and Test-Time Scaling of Large Language Models Xiaobao Wu LRM 72 1 0 05 May 2025
A Domain-Agnostic Scalable AI Safety Ensuring Framework Beomjun Kim Kangyeon Kim Sunwoo Kim Heejin Ahn 55 0 0 29 Apr 2025
CachePrune: Neural-Based Attribution Defense Against Indirect Prompt Injection Attacks Rui Wang Junda Wu Yu Xia Tong Yu R. Zhang Ryan Rossi Lina Yao Julian McAuley AAML SILM 51 0 0 29 Apr 2025
JailbreaksOverTime: Detecting Jailbreak Attacks Under Distribution Shift Julien Piet Xiao Huang Dennis Jacob Annabella Chow Maha Alrashed Geng Zhao Zhanhao Hu Chawin Sitawarin Basel Alomair David A. Wagner AAML 70 0 0 28 Apr 2025
Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors Ren-Wei Liang Chin-Ting Hsu Chan-Hung Yu Saransh Agrawal Shih-Cheng Huang Shang-Tse Chen Kuan-Hao Huang Shao-Hua Sun 81 0 0 27 Apr 2025
Anyprefer: An Agentic Framework for Preference Data Synthesis Yiyang Zhou Zekun Wang Tianle Wang Shangyu Xing Peng Xia ... Chetan Bansal Weitong Zhang Ying Wei Joey Tianyi Zhou Huaxiu Yao 61 0 0 27 Apr 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong Xuzhao Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Tianwei Zhang ALM ELM 86 1 0 26 Apr 2025
RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models Bang An Shiyue Zhang Mark Dredze 61 0 0 25 Apr 2025
Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control Hannah Cyberey David E. Evans LLMSV 76 0 0 23 Apr 2025
aiXamine: Simplified LLM Safety and Security Fatih Deniz Dorde Popovic Yazan Boshmaf Euisuh Jeong M. Ahmad Sanjay Chawla Issa M. Khalil ELM 80 0 0 21 Apr 2025
Alleviating the Fear of Losing Alignment in LLM Fine-tuning Kang Yang Guanhong Tao X. Chen Jun Xu 36 0 0 13 Apr 2025
Information-Theoretic Reward Decomposition for Generalizable RLHF Liyuan Mao Haoran Xu Amy Zhang Weinan Zhang Chenjia Bai 33 0 0 08 Apr 2025
Following the Whispers of Values: Unraveling Neural Mechanisms Behind Value-Oriented Behaviors in LLMs Ling Hu Yuemei Xu Xiaoyang Gu Letao Han 28 0 0 07 Apr 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian-Yu Guan Jian Wu J. Li Chuanqi Cheng Wei Yu Wu LM&MA 74 0 0 21 Mar 2025
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment J. Li Jian-Yu Guan Songhao Wu Wei Yu Wu Rui Yan 64 1 0 19 Mar 2025
PEO: Improving Bi-Factorial Preference Alignment with Post-Training Policy Extrapolation Yuxuan Liu 45 0 0 03 Mar 2025
Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs Abdelrahman Abouelenin Atabak Ashfaq Adam Atkinson Hany Awadalla Nguyen Bach ... Ishmam Zabir Yunan Zhang Li Zhang Yuhang Zhang Xiren Zhou MoE SyDa 70 24 0 03 Mar 2025
Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable Tiansheng Huang Sihao Hu Fatih Ilhan Selim Furkan Tekin Zachary Yahn Yichang Xu Ling Liu 53 9 0 01 Mar 2025
Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework Kaishuai Xu Tiezheng YU Wenjun Hou Yi Cheng Liangyou Li Xin Jiang Lifeng Shang Qiang Liu Wenjie Li ELM 66 0 0 26 Feb 2025
Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation Shuo Tang Xianghe Pang Zexi Liu Bohan Tang Rui Ye Xiaowen Dong Yunhong Wang Yanfeng Wang S. Chen SyDa LLMAG 129 4 0 21 Feb 2025
SafeDialBench: A Fine-Grained Safety Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attacks Hongye Cao Yanming Wang Sijia Jing Ziyue Peng Zhixin Bai ... Yang Gao Fanyu Meng Xi Yang Chao Deng Junlan Feng AAML 41 0 0 16 Feb 2025
SEER: Self-Explainability Enhancement of Large Language Models' Representations Guanxu Chen Dongrui Liu Tao Luo Jing Shao LRM MILM 67 1 0 07 Feb 2025
Out-of-Distribution Detection using Synthetic Data Generation Momin Abbas Muneeza Azmat R. Horesh Mikhail Yurochkin 47 1 0 05 Feb 2025
Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models Haoran Ye T. Zhang Yuhang Xie Liyuan Zhang Yuanyi Ren Xin Zhang Guojie Song PILM 81 0 0 04 Feb 2025
GuardReasoner: Towards Reasoning-based LLM Safeguards Yue Liu Hongcheng Gao Shengfang Zhai Jun-Xiong Xia Tianyi Wu Zhiwei Xue Y. Chen Kenji Kawaguchi Jiaheng Zhang Bryan Hooi AI4TS LRM 131 14 0 30 Jan 2025
Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback Yafu Li Xuyang Hu Xiaoye Qu Linjie Li Yu-Xi Cheng 53 3 0 22 Jan 2025
Understanding the Logic of Direct Preference Alignment through Logic Kyle Richardson Vivek Srikumar Ashish Sabharwal 85 2 0 23 Dec 2024
Reinforcement Learning Enhanced LLMs: A Survey Shuhe Wang Shengyu Zhang Jingyang Zhang Runyi Hu Xiaoya Li Tianwei Zhang Jiwei Li Fei Wu G. Wang Eduard H. Hovy OffRL 134 7 0 05 Dec 2024
JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit Zeqing He Zhibo Wang Zhixuan Chu Huiyu Xu Rui Zheng Kui Ren Chun Chen 54 3 0 17 Nov 2024
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset Khaoula Chehbouni Jonathan Colaço-Carr Yash More Jackie CK Cheung G. Farnadi 78 0 0 12 Nov 2024
Diversity Helps Jailbreak Large Language Models Weiliang Zhao Daniel Ben-Levi Wei Hao Junfeng Yang Chengzhi Mao AAML 147 0 0 06 Nov 2024
SQL Injection Jailbreak: A Structural Disaster of Large Language Models Jiawei Zhao Kejiang Chen Wenbo Zhang Nenghai Yu AAML 40 0 0 03 Nov 2024
L3Ms -- Lagrange Large Language Models Guneet S. Dhillon Xingjian Shi Yee Whye Teh Alex Smola 147 0 0 28 Oct 2024
Enhancing Safety in Reinforcement Learning with Human Feedback via Rectified Policy Optimization Xiyue Peng Hengquan Guo Jiawei Zhang Dongqing Zou Ziyu Shao Honghao Wei Xin Liu 44 0 0 25 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 48 4 0 23 Oct 2024
WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models Jinghan Jia Jiancheng Liu Yihua Zhang Parikshit Ram Nathalie Baracaldo Sijia Liu MU 35 2 0 23 Oct 2024
Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks Samuele Poppi Zheng-Xin Yong Yifei He Bobbie Chern Han Zhao Aobo Yang Jianfeng Chi AAML 45 14 0 23 Oct 2024
CLEAR: Character Unlearning in Textual and Visual Modalities Alexey Dontsov Dmitrii Korzh Alexey Zhavoronkin Boris Mikheev Denis Bobkov Aibek Alanov Oleg Y. Rogov Ivan V. Oseledets Elena Tutubalina AILaw VLM MU 66 5 0 23 Oct 2024
Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment Mingzhi Wang Chengdong Ma Qizhi Chen Linjian Meng Yang Han Jiancong Xiao Zhaowei Zhang Jing Huo Weijie Su Yaodong Yang 32 4 0 22 Oct 2024
AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignment Pankayaraj Pathmanathan Udari Madhushani Sehwag Michael-Andrei Panaitescu-Liess Furong Huang SILM AAML 43 0 0 15 Oct 2024
On Calibration of LLM-based Guard Models for Reliable Content Moderation Hongfu Liu Hengguan Huang Hao Wang Xiangming Gu Ye Wang 55 2 0 14 Oct 2024
Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation Guozhi Liu Weiwei Lin Tiansheng Huang Ruichao Mo Qi Mu Li Shen AAML 63 10 0 13 Oct 2024
Taming Overconfidence in LLMs: Reward Calibration in RLHF Jixuan Leng Chengsong Huang Banghua Zhu Jiaxin Huang 34 7 0 13 Oct 2024
Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements Jingyu Zhang Ahmed Elgohary Ahmed Magooda Daniel Khashabi Benjamin Van Durme 138 2 0 11 Oct 2024
GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment Yuancheng Xu Udari Madhushani Sehwag Alec Koppel Sicheng Zhu Bang An Furong Huang Sumitra Ganesh 55 6 0 10 Oct 2024
Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond Shanshan Han 84 1 0 09 Oct 2024
Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learning Hao Ma Tianyi Hu Zhiqiang Pu Boyin Liu Xiaolin Ai Yanyan Liang Min Chen 50 3 0 08 Oct 2024