Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions

14 September 2023

Federico Bianchi

Mirac Suzgun

Giuseppe Attanasio

Paul Röttger

Dan Jurafsky

Tatsunori Hashimoto

Papers citing "Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions"

50 / 140 papers shown

Title
Analysing Safety Risks in LLMs Fine-Tuned with Pseudo-Malicious Cyber Security Data Adel ElZemity Budi Arief Shujun Li 31 0 0 15 May 2025
FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning Zhehao Zhang Weijie Xu Fanyou Wu Chandan K. Reddy 29 0 0 12 May 2025
Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety Zihan Guan Mengxuan Hu Ronghang Zhu Sheng Li Anil Vullikanti AAML 31 0 0 11 May 2025
Safety Pretraining: Toward the Next Generation of Safe AI Pratyush Maini Sachin Goyal Dylan Sam Alex Robey Yash Savani Yiding Jiang Andy Zou Zacharcy C. Lipton J. Zico Kolter 63 0 0 23 Apr 2025
Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control Hannah Cyberey David E. Evans LLMSV 76 0 0 23 Apr 2025
SaRO: Enhancing LLM Safety through Reasoning-based Alignment Yutao Mou Yuxiao Luo Shikun Zhang Wei Ye LLMSV LRM 36 0 0 13 Apr 2025
AttentionDefense: Leveraging System Prompt Attention for Explainable Defense Against Novel Jailbreaks Charlotte Siska Anush Sankaran AAML 45 0 0 10 Apr 2025
LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation Juzheng Zhang Jiacheng You Ashwinee Panda Tom Goldstein MoMe 53 0 0 10 Apr 2025
$PiCo: Jailbreaking Multimodal Large Language Models via $\textbf{Pi}$ctorial $\textbf{Co}$de Contextualization$ PiCo: Jailbreaking Multimodal Large Language Models via $\textbf{Pi}$ ctorial $\textbf{Co}$ de Contextualization Aofan Liu Lulu Tang Ting Pan Yuguo Yin Bin Wang Ao Yang MLLM AAML 45 0 0 02 Apr 2025
Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks Jiawei Wang Yushen Zuo Yuanjun Chai Ziqiang Liu Yichen Fu Yichun Feng Kin-Man Lam AAML VLM 42 0 0 02 Apr 2025
Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models Pin-Yu Chen Han Shen Payel Das Tianyi Chen 50 0 0 24 Mar 2025
Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities Weixiang Zhao Xingyu Sui Jiahe Guo Yulin Hu Yang Deng Yanyan Zhao Bing Qin Wanxiang Che Tat-Seng Chua Ting Liu ELM LRM 59 4 0 23 Mar 2025
Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior Shri Kiran Srinivasan Xinpeng Wang Guangyao Zhai Nassir Navab Barbara Plank LLMAG 51 0 0 22 Mar 2025
SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging Aladin Djuhera S. Kadhe Farhan Ahmed Syed Zawad Holger Boche MoMe 51 0 0 21 Mar 2025
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-tuning Yiwei Chen Yuguang Yao Yihua Zhang Bingquan Shen Gaowen Liu Sijia Liu AAML MU 58 1 0 14 Mar 2025
Adversarial Training for Multimodal Large Language Models against Jailbreak Attacks Liming Lu Shuchao Pang Siyuan Liang Haotian Zhu Xiyu Zeng Aishan Liu Yunhuai Liu Yongbin Zhou AAML 51 1 0 05 Mar 2025
Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs Abdelrahman Abouelenin Atabak Ashfaq Adam Atkinson Hany Awadalla Nguyen Bach ... Ishmam Zabir Yunan Zhang Li Zhang Yuhang Zhang Xiren Zhou MoE SyDa 70 24 0 03 Mar 2025
Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs Weixiang Zhao Yulin Hu Yang Deng Jiahe Guo Xingyu Sui ... An Zhang Yanyan Zhao Bing Qin Tat-Seng Chua Ting Liu 62 1 0 28 Feb 2025
The Call for Socially Aware Language Technologies Diyi Yang Dirk Hovy David Jurgens Barbara Plank VLM 61 11 0 24 Feb 2025
AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement Zhexin Zhang Leqi Lei Junxiao Yang Xijie Huang Yida Lu ... Xianqi Lei C. Pan Lei Sha Hairu Wang Minlie Huang AAML 48 0 0 24 Feb 2025
Control Illusion: The Failure of Instruction Hierarchies in Large Language Models Yilin Geng Hao Li Honglin Mu Xudong Han Timothy Baldwin Omri Abend Eduard H. Hovy Lea Frermann 41 2 0 21 Feb 2025
Refining Positive and Toxic Samples for Dual Safety Self-Alignment of LLMs with Minimal Human Interventions Jingxin Xu Guoshun Nan Sheng Guan Sicong Leng Yong-Jin Liu Zixiao Wang Yuyang Ma Zhili Zhou Yanzhao Hou Xiaofeng Tao LM&MA 55 0 0 08 Feb 2025
Leveraging Reasoning with Guidelines to Elicit and Utilize Knowledge for Enhancing Safety Alignment Haoyu Wang Zeyu Qin Li Shen Xueqian Wang Minhao Cheng Dacheng Tao 99 2 0 06 Feb 2025
Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation Yishuo Wang Tiansheng Huang Li Shen H. Yao Haotian Luo Rui Liu Naiqiang Tan Jiaxing Huang Dacheng Tao AAML MoMe CLL 111 2 0 30 Jan 2025
You Can't Eat Your Cake and Have It Too: The Performance Degradation of LLMs with Jailbreak Defense Wuyuao Mai Geng Hong Pei Chen Xudong Pan Baojun Liu Y. Zhang Haixin Duan Min Yang AAML 68 1 0 21 Jan 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 82 44 0 20 Jan 2025
Can Safety Fine-Tuning Be More Principled? Lessons Learned from Cybersecurity David Williams-King Linh Le Adam Oberman Yoshua Bengio AAML 56 0 0 19 Jan 2025
LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs LLM-jp Akiko Aizawa Eiji Aramaki Bowen Chen Fei Cheng ... Yuya Yamamoto Yusuke Yamauchi Hitomi Yanaka Rio Yokota Koichiro Yoshino 57 14 0 31 Dec 2024
Enhancing AI Safety Through the Fusion of Low Rank Adapters Satya Swaroop Gudipudi Sreeram Vipparla Harpreet Singh Shashwat Goel Ponnurangam Kumaraguru MoMe AAML 44 2 0 30 Dec 2024
Chained Tuning Leads to Biased Forgetting Megan Ung Alicia Sun Samuel J. Bell Bhaktipriya Radharapu Levent Sagun Adina Williams CLL KELM 89 0 0 21 Dec 2024
Phi-4 Technical Report Marah Abdin J. Aneja Harkirat Singh Behl Sébastien Bubeck Ronen Eldan ... Rachel A. Ward Yue Wu Dingli Yu Cyril Zhang Yi Zhang ALM SyDa 96 86 0 12 Dec 2024
PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning Shenghui Li Edith C. H. Ngai Fanghua Ye Thiemo Voigt SILM 90 6 0 28 Nov 2024
$H^3$ Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs Selim Furkan Tekin Fatih Ilhan Tiansheng Huang Sihao Hu Zachary Yahn Ling Liu MoMe 86 3 0 26 Nov 2024
Safe to Serve: Aligning Instruction-Tuned Models for Safety and Helpfulness Avinash Amballa Durga Sandeep Saluru Gayathri Akkinapalli Abhishek Sureddy Akshay Kumar Sureddy ALM 90 0 0 26 Nov 2024
Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspective Jean Marie Tshimula Xavier Ndona D'Jeff K. Nkashama Pierre Martin Tardif F. Kabanza Marc Frappier Shengrui Wang SILM 84 0 0 25 Nov 2024
Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey Xuannan Liu Xing Cui Peipei Li Zekun Li Huaibo Huang Shuhan Xia Miaoxuan Zhang Yueying Zou Ran He AAML 65 6 0 14 Nov 2024
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset Khaoula Chehbouni Jonathan Colaço-Carr Yash More Jackie CK Cheung G. Farnadi 78 0 0 12 Nov 2024
Rule Based Rewards for Language Model Safety Tong Mu Alec Helyar Johannes Heidecke Joshua Achiam Andrea Vallone Ian Kivlichan Molly Lin Alex Beutel John Schulman Lilian Weng ALM 42 35 0 02 Nov 2024
Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari AAML 36 3 0 31 Oct 2024
Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring Honglin Mu Han He Yuxin Zhou Yunlong Feng Yang Xu ... Zeming Liu Xudong Han Qi Shi Qingfu Zhu Wanxiang Che AAML 35 1 0 28 Oct 2024
BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks Yunhan Zhao Xiang Zheng Lin Luo Yige Li Xingjun Ma Yu-Gang Jiang VLM AAML 60 3 0 28 Oct 2024
Enhancing Safety in Reinforcement Learning with Human Feedback via Rectified Policy Optimization Xiyue Peng Hengquan Guo Jiawei Zhang Dongqing Zou Ziyu Shao Honghao Wei Xin Liu 44 0 0 25 Oct 2024
Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Models He Cao Weidi Luo Zijing Liu Yu Wang Bing Feng Yuan Yao Yuan Yao Yu Li AAML 56 1 0 23 Oct 2024
POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization Batuhan K. Karaman Ishmam Zabir Alon Benhaim Vishrav Chaudhary M. Sabuncu Xia Song AI4CE 35 0 0 16 Oct 2024
Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models Shicheng Xu Liang Pang Yunchang Zhu Huawei Shen Xueqi Cheng MLLM 36 1 0 16 Oct 2024
Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Models Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari AAML 36 1 0 15 Oct 2024
Locking Down the Finetuned LLMs Safety Minjun Zhu Linyi Yang Yifan Wei Ningyu Zhang Yue Zhang 34 8 0 14 Oct 2024
Safety-Aware Fine-Tuning of Large Language Models Hyeong Kyu Choi Xuefeng Du Yixuan Li 45 11 0 13 Oct 2024
Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation Guozhi Liu Weiwei Lin Tiansheng Huang Ruichao Mo Qi Mu Li Shen AAML 63 10 0 13 Oct 2024
JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework Fan Liu Yue Feng Zhao Xu Lixin Su Xinyu Ma Dawei Yin Hao Liu ELM 32 7 0 11 Oct 2024