v1v2v3 (latest)

Removing RLHF Protections in GPT-4 via Fine-Tuning

9 November 2023

Tatsunori Hashimoto

Papers citing "Removing RLHF Protections in GPT-4 via Fine-Tuning"

50 / 81 papers shown

Title
Probing the Robustness of Large Language Models Safety to Latent Perturbations Tianle Gu Kexin Huang Zongqi Wang Yixu Wang Jie Li Yuanqi Yao Yang Yao Yujiu Yang Yan Teng Yingchun Wang AAML LLMSV 44 0 0 19 Jun 2025
Vulnerability-Aware Alignment: Mitigating Uneven Forgetting in Harmful Fine-Tuning Liang Chen Xueting Han Li Shen Jing Bai Kam-Fai Wong AAML 82 0 0 04 Jun 2025
Model Immunization from a Condition Number Perspective Amber Yijia Zheng Cedar Site Bai Brian Bullins Raymond A. Yeh MedIm 21 0 0 29 May 2025
Refusal Direction is Universal Across Safety-Aligned Languages Xinpeng Wang Mingyang Wang Yihong Liu Hinrich Schutze Barbara Plank 230 1 0 22 May 2025
CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning Biao Yi Tiansheng Huang Baolei Zhang Tong Li Lihai Nie Zheli Liu Li Shen MU AAML 98 0 0 22 May 2025
Shape it Up! Restoring LLM Safety during Finetuning ShengYun Peng Pin-Yu Chen Jianfeng Chi Seongmin Lee Duen Horng Chau 70 0 0 22 May 2025
Trust Me, I Can Handle It: Self-Generated Adversarial Scenario Extrapolation for Robust Language Models Md Rafi Ur Rashid Vishnu Asutosh Dasu Ye Wang Gang Tan Shagufta Mehnaz AAML ELM 109 0 0 20 May 2025
Safety Subspaces are Not Distinct: A Fine-Tuning Case Study Kaustubh Ponkshe Shaan Shah Raghav Singhal Praneeth Vepakomma 126 0 0 20 May 2025
Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets Ning Lu Shengcai Liu Jiahao Wu Weiyu Chen Zhirui Zhang Yew-Soon Ong Qi Wang Ke Tang 108 3 0 17 May 2025
Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety Zihan Guan Mengxuan Hu Ronghang Zhu Sheng Li Anil Vullikanti AAML 83 3 0 11 May 2025
Fight Fire with Fire: Defending Against Malicious RL Fine-Tuning via Reward Neutralization Wenjun Cao AAML 83 0 0 07 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Jen-tse Huang Joey Tianyi Zhou AAML MU 285 4 0 01 May 2025
Emergence of Computational Structure in a Neural Network Physics Simulator Rohan Hitchcock Gary W. Delaney J. Manton Richard Scalzo Jingge Zhu 63 0 0 16 Apr 2025
AttentionDefense: Leveraging System Prompt Attention for Explainable Defense Against Novel Jailbreaks Charlotte Siska Anush Sankaran AAML 90 1 0 10 Apr 2025
SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging Aladin Djuhera S. Kadhe Farhan Ahmed Syed Zawad Holger Boche MoMe 92 4 0 21 Mar 2025
AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations Dillon Bowen Ann-Kathrin Dombrowski Adam Gleave Chris Cundy ELM 75 0 0 17 Mar 2025
Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities Zora Che Stephen Casper Robert Kirk Anirudh Satheesh Stewart Slocum ... Zikui Cai Bilal Chughtai Y. Gal Furong Huang Dylan Hadfield-Menell MU AAML ELM 185 7 0 03 Feb 2025
Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation Yun Wang Tiansheng Huang Li Shen Huanjin Yao Haotian Luo Rui Liu Naiqiang Tan Jiaxing Huang Dacheng Tao AAML MoMe CLL 209 4 0 30 Jan 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 133 59 0 20 Jan 2025
Enhancing AI Safety Through the Fusion of Low Rank Adapters Satya Swaroop Gudipudi Sreeram Vipparla Harpreet Singh Shashwat Goel Ponnurangam Kumaraguru MoMe AAML 86 3 0 30 Dec 2024
On Evaluating the Durability of Safeguards for Open-Weight LLMs Xiangyu Qi Boyi Wei Nicholas Carlini Yangsibo Huang Tinghao Xie Luxi He Matthew Jagielski Milad Nasr Prateek Mittal Peter Henderson AAML 137 22 0 10 Dec 2024
PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning Shenghui Li Edith C.H. Ngai Fanghua Ye Thiemo Voigt SILM 197 6 0 28 Nov 2024
Focus On This, Not That! Steering LLMs with Adaptive Feature Specification Tom A. Lamb Adam Davies Alasdair Paren Philip Torr Francesco Pinto 127 0 0 30 Oct 2024
The effect of fine-tuning on language model toxicity Will Hawkins Brent Mittelstadt Chris Russell 66 5 0 21 Oct 2024
Multi-round jailbreak attack on large language models Yihua Zhou Xiaochuan Shi AAML 59 0 0 15 Oct 2024
SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimization Akrit Mudvari Yuang Jiang Leandros Tassiulas 71 6 0 14 Oct 2024
Locking Down the Finetuned LLMs Safety Minjun Zhu Linyi Yang Yifan Wei Ningyu Zhang Yue Zhang 108 14 0 14 Oct 2024
Jailbreak Instruction-Tuned LLMs via end-of-sentence MLP Re-weighting Yifan Luo Zhennan Zhou Meitan Wang Bin Dong 95 1 0 14 Oct 2024
Safety-Aware Fine-Tuning of Large Language Models Hyeong Kyu Choi Xuefeng Du Yixuan Li 96 19 0 13 Oct 2024
Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements Jingyu Zhang Ahmed Elgohary Ahmed Magooda Daniel Khashabi Benjamin Van Durme 475 8 0 11 Oct 2024
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 223 35 0 11 Oct 2024
Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks Mathis Pink Vy A. Vo Qinyuan Wu Jianing Mu Javier S. Turek Uri Hasson K. A. Norman Sebastian Michelmann Alexander G. Huth Mariya Toneva 106 2 0 10 Oct 2024
Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey Tiansheng Huang Sihao Hu Fatih Ilhan Selim Furkan Tekin Ling Liu AAML 140 46 0 26 Sep 2024
Recent Advances in Attack and Defense Approaches of Large Language Models Jing Cui Yishi Xu Zhewei Huang Shuchang Zhou Jianbin Jiao Junge Zhang PILM AAML 133 2 0 05 Sep 2024
The Dark Side of Human Feedback: Poisoning Large Language Models via User Inputs Bocheng Chen Hanqing Guo Guangjing Wang Yuanda Wang Qiben Yan AAML 106 5 0 01 Sep 2024
Acceptable Use Policies for Foundation Models Kevin Klyman 69 17 0 29 Aug 2024
Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Tiansheng Huang Gautam Bhattacharya Pratik Joshi Josh Kimball Ling Liu AAML MoMe 111 30 0 18 Aug 2024
WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Models Guitao Chen Yunshen Wang Hongye Sun Guang Chen MU 67 1 0 18 Aug 2024
Mission Impossible: A Statistical Perspective on Jailbreaking LLMs Jingtong Su Mingyu Lee SangKeun Lee 93 12 0 02 Aug 2024
Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models Yingkai Dong Xiangtao Meng Ning Yu Zheng Li Shanqing Guo LLMAG 121 17 0 01 Aug 2024
Tamper-Resistant Safeguards for Open-Weight LLMs Rishub Tamirisa Bhrugu Bharathi Long Phan Andy Zhou Alice Gatti ... Andy Zou Dawn Song Bo Li Dan Hendrycks Mantas Mazeika AAML MU 133 63 0 01 Aug 2024
The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Models Zihui Wu Haichang Gao Jianping He Ping Wang 112 10 0 25 Jul 2024
Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs) Apurv Verma Satyapriya Krishna Sebastian Gehrmann Madhavan Seshadri Anu Pradhan Tom Ault Leslie Barrett David Rabinowitz John Doucette Nhathai Phan 129 15 0 20 Jul 2024
Jailbreak Attacks and Defenses Against Large Language Models: A Survey Sibo Yi Yule Liu Zhen Sun Tianshuo Cong Xinlei He Jiaxing Song Ke Xu Qi Li AAML 124 111 0 05 Jul 2024
Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation Danny Halawi Alexander Wei Eric Wallace Tony T. Wang Nika Haghtalab Jacob Steinhardt SILM AAML 103 36 0 28 Jun 2024
JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models Haibo Jin Leyang Hu Xinuo Li Peiyan Zhang Chonghan Chen Jun Zhuang Haohan Wang PILM 103 32 0 26 Jun 2024
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs Ashwinee Panda Berivan Isik Xiangyu Qi Sanmi Koyejo Tsachy Weissman Prateek Mittal MoMe 139 16 0 24 Jun 2024
Refusal in Language Models Is Mediated by a Single Direction Andy Arditi Oscar Obeso Aaquib Syed Daniel Paleka Nina Panickssery Wes Gurnee Neel Nanda 169 218 0 17 Jun 2024
Towards Lifelong Learning of Large Language Models: A Survey Junhao Zheng Shengjie Qiu Chengming Shi Qianli Ma KELM CLL 83 28 0 10 Jun 2024
Safety Alignment Should Be Made More Than Just a Few Tokens Deep Xiangyu Qi Ashwinee Panda Kaifeng Lyu Xiao Ma Subhrajit Roy Ahmad Beirami Prateek Mittal Peter Henderson 120 142 0 10 Jun 2024