Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models

4 October 2023

Dahua Lin

Papers citing "Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models"

43 / 43 papers shown

Title
CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning Biao Yi Tiansheng Huang Baolei Zhang Tong Li Lihai Nie Zheli Liu Li Shen MU AAML 21 0 0 22 May 2025
Safety Subspaces are Not Distinct: A Fine-Tuning Case Study Kaustubh Ponkshe Shaan Shah Raghav Singhal Praneeth Vepakomma 28 0 0 20 May 2025
JULI: Jailbreak Large Language Models by Self-Introspection Jesson Wang Zhanhao Hu David Wagner 33 0 0 17 May 2025
Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets Ning Lu Shengcai Liu Jiahao Wu Weiyu Chen Zhirui Zhang Yew-Soon Ong Qi Wang Ke Tang 28 1 0 17 May 2025
Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety Zihan Guan Mengxuan Hu Ronghang Zhu Sheng Li Anil Vullikanti AAML 38 0 0 11 May 2025
Teaching Models to Understand (but not Generate) High-risk Data Ryan Yixiang Wang Matthew Finlayson Luca Soldaini Swabha Swayamdipta Robin Jia 246 0 0 05 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Jen-tse Huang Joey Tianyi Zhou AAML MU 99 4 0 01 May 2025
Alleviating the Fear of Losing Alignment in LLM Fine-tuning Kang Yang Guanhong Tao X. Chen Jun Xu 51 0 0 13 Apr 2025
Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs Weixiang Zhao Yulin Hu Yang Deng Jiahe Guo Xingyu Sui ... An Zhang Yanyan Zhao Bing Qin Tat-Seng Chua Ting Liu 91 3 0 28 Feb 2025
Single-pass Detection of Jailbreaking Input in Large Language Models Leyla Naz Candogan Yongtao Wu Elias Abad Rocamora Grigorios G. Chrysos Volkan Cevher AAML 53 0 0 24 Feb 2025
Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities Zora Che Stephen Casper Robert Kirk Anirudh Satheesh Stewart Slocum ... Zikui Cai Bilal Chughtai Y. Gal Furong Huang Dylan Hadfield-Menell MU AAML ELM 89 2 0 03 Feb 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 84 50 0 20 Jan 2025
Steering Language Model Refusal with Sparse Autoencoders Kyle O'Brien David Majercak Xavier Fernandes Richard Edgar Jingya Chen Harsha Nori Harsha Nori Dean Carignan Eric Horvitz Forough Poursabzi-Sangde LLMSV 94 12 0 18 Nov 2024
Focus On This, Not That! Steering LLMs With Adaptive Feature Specification Tom A. Lamb Adam Davies Alasdair Paren Philip Torr Francesco Pinto 75 0 0 30 Oct 2024
Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense Rui Min Zeyu Qin Nevin L. Zhang Li Shen Minhao Cheng AAML 52 4 0 13 Oct 2024
Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation Guozhi Liu Weiwei Lin Tiansheng Huang Ruichao Mo Qi Mu Li Shen AAML 73 11 0 13 Oct 2024
PII-Scope: A Comprehensive Study on Training Data PII Extraction Attacks in LLMs Krishna Kanth Nakka Ahmed Frikha Ricardo Mendes Xue Jiang Xuebing Zhou 38 3 0 09 Oct 2024
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Jing Jiang Min Lin 53 8 0 09 Oct 2024
An Adversarial Perspective on Machine Unlearning for AI Safety Jakub Łucki Boyi Wei Yangsibo Huang Peter Henderson F. Tramèr Javier Rando MU AAML 86 36 0 26 Sep 2024
Recent Advances in Attack and Defense Approaches of Large Language Models Jing Cui Yishi Xu Zhewei Huang Shuchang Zhou Jianbin Jiao Junge Zhang PILM AAML 64 1 0 05 Sep 2024
Learning to Ask: When LLM Agents Meet Unclear Instruction Wenxuan Wang Juluan Shi Chaozheng Wang Cheryl Lee Chaozheng Wang Cheryl Lee Youliang Yuan Jen-tse Huang Wenxiang Jiao Michael R. Lyu LLMAG 44 8 0 31 Aug 2024
Acceptable Use Policies for Foundation Models Kevin Klyman 43 14 0 29 Aug 2024
Measuring Human Contribution in AI-Assisted Content Generation Yueqi Xie Tao Qi Jingwei Yi Ryan Whalen Junming Huang Qian Ding Yu Xie Xing Xie Fangzhao Wu Fangzhao Wu 47 1 0 27 Aug 2024
Tamper-Resistant Safeguards for Open-Weight LLMs Rishub Tamirisa Bhrugu Bharathi Long Phan Andy Zhou Alice Gatti ... Andy Zou Dawn Song Bo Li Dan Hendrycks Mantas Mazeika AAML MU 72 48 0 01 Aug 2024
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training Youliang Yuan Wenxiang Jiao Wenxuan Wang Jen-tse Huang Jiahao Xu Tian Liang Pinjia He Zhaopeng Tu 47 23 0 12 Jul 2024
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models Seungone Kim Juyoung Suk Ji Yong Cho Shayne Longpre Chaeeun Kim ... Sean Welleck Graham Neubig Moontae Lee Kyungjae Lee Minjoon Seo ELM ALM LM&MA 114 35 0 09 Jun 2024
A Survey of Language-Based Communication in Robotics William Hunt Sarvapali D. Ramchurn Mohammad D. Soorati LM&Ro 101 13 0 06 Jun 2024
Teams of LLM Agents can Exploit Zero-Day Vulnerabilities Richard Fang Antony Kellermann Akul Gupta Qiusi Zhan Richard Fang R. Bindu Daniel Kang LLMAG 45 31 0 02 Jun 2024
Stress-Testing Capability Elicitation With Password-Locked Models Ryan Greenblatt Fabien Roger Dmitrii Krasheninnikov David M. Krueger 58 14 0 29 May 2024
Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models Chia-Yi Hsu Yu-Lin Tsai Chih-Hsun Lin Pin-Yu Chen Chia-Mu Yu Chun-ying Huang 75 42 0 27 May 2024
Societal Adaptation to Advanced AI Jamie Bernardi Gabriel Mukobi Hilary Greaves Lennart Heim Markus Anderljung 51 5 0 16 May 2024
When LLMs Meet Cybersecurity: A Systematic Literature Review Jie Zhang Haoyu Bu Hui Wen Yu Chen Lun Li Hongsong Zhu 75 36 0 06 May 2024
On the Societal Impact of Open Foundation Models Sayash Kapoor Rishi Bommasani Kevin Klyman Shayne Longpre Ashwin Ramaswami ... Victor Storchan Daniel Zhang Daniel E. Ho Percy Liang Arvind Narayanan 31 56 0 27 Feb 2024
Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment Jiong Wang Jiazhao Li Yiquan Li Xiangyu Qi Junjie Hu Yixuan Li P. McDaniel Muhao Chen Bo Li Chaowei Xiao AAML SILM 45 19 0 22 Feb 2024
Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content Federico Bianchi James Zou 32 4 0 21 Feb 2024
Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning Zhaorui Yang Tianyu Pang Hao Feng Han Wang Wei Chen Minfeng Zhu Qian Liu ALM 42 44 0 21 Feb 2024
LLM Agents can Autonomously Hack Websites Richard Fang R. Bindu Akul Gupta Qiusi Zhan Daniel Kang LLMAG 27 54 0 06 Feb 2024
Navigating the OverKill in Large Language Models Chenyu Shi Xiao Wang Qiming Ge Songyang Gao Xianjun Yang Tao Gui Qi Zhang Xuanjing Huang Xun Zhao Dahua Lin 32 11 0 31 Jan 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 49 82 0 25 Jan 2024
Exploiting Novel GPT-4 APIs Kellin Pelrine Mohammad Taufeeque Michal Zajkac Euan McLean Adam Gleave SILM 39 20 0 21 Dec 2023
LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B Simon Lermen Charlie Rogers-Smith Jeffrey Ladish ALM 38 85 0 31 Oct 2023
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT Jules White Quchen Fu Sam Hays Michael Sandborn Carlos Olea Henry Gilbert Ashraf Elnashar Jesse Spencer-Smith Douglas C. Schmidt LLMAG 94 1,066 0 21 Feb 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 457 12,345 0 04 Mar 2022