v1v2 (latest)

Refusal in Language Models Is Mediated by a Single Direction

17 June 2024

Nina Panickssery

Papers citing "Refusal in Language Models Is Mediated by a Single Direction"

50 / 55 papers shown

Title
Understanding (Un)Reliability of Steering Vectors in Language Models Joschka Braun Carsten Eickhoff David M. Krueger Seyed Ali Bahrainian Dmitrii Krasheninnikov LLMSV 69 1 0 28 May 2025
Do BERT-Like Bidirectional Models Still Perform Better on Text Classification in the Era of LLMs? Junyan Zhang Yiming Huang Shuliang Liu Yubo Gao Xuming Hu 151 0 0 23 May 2025
When Do LLMs Admit Their Mistakes? Understanding the Role of Model Belief in Retraction Yuqing Yang Robin Jia KELM LRM 93 1 0 22 May 2025
Refusal Direction is Universal Across Safety-Aligned Languages Xinpeng Wang Mingyang Wang Yihong Liu Hinrich Schutze Barbara Plank 212 1 0 22 May 2025
Safety Subspaces are Not Distinct: A Fine-Tuning Case Study Kaustubh Ponkshe Shaan Shah Raghav Singhal Praneeth Vepakomma 106 0 0 20 May 2025
Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control Hannah Cyberey David Evans LLMSV 149 3 0 23 Apr 2025
The Geometry of Self-Verification in a Task-Specific Reasoning Model Andrew Lee Lihao Sun Chris Wendler Fernanda Viégas Martin Wattenberg LRM 142 1 0 19 Apr 2025
Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations Ziwei Ji L. Yu Yeskendir Koishekenov Yejin Bang Anthony Hartshorn Alan Schelten Cheng Zhang Pascale Fung Nicola Cancedda 77 5 0 18 Mar 2025
ASIDE: Architectural Separation of Instructions and Data in Language Models Egor Zverev Evgenii Kortukov Alexander Panfilov Soroush Tabesh Alexandra Volkova Sebastian Lapuschkin Wojciech Samek Christoph H. Lampert AAML 104 2 0 13 Mar 2025
Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy Abe Bohan Hou H. Du Yichen Wang Jingyu Zhang Zixiao Wang Paul Pu Liang Daniel Khashabi Lauren M. Gardner Tianxing He 117 1 0 12 Mar 2025
Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models Thomas Winninger Boussad Addad Katarzyna Kapusta AAML 115 1 0 08 Mar 2025
Linear Representations of Political Perspective Emerge in Large Language Models Junsol Kim James Evans Aaron Schein 124 6 0 03 Mar 2025
Sensing and Steering Stereotypes: Extracting and Applying Gender Representation Vectors in LLMs Hannah Cyberey Yangfeng Ji David Evans LLMSV 129 1 0 27 Feb 2025
SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention Jiaqi Wu Chen Chen Chunyan Hou Xiaojie Yuan AAML 104 0 0 21 Feb 2025
Activation Steering in Neural Theorem Provers Shashank Kirtania LLMSV 459 0 0 21 Feb 2025
Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities Zora Che Stephen Casper Robert Kirk Anirudh Satheesh Stewart Slocum ... Zikui Cai Bilal Chughtai Y. Gal Furong Huang Dylan Hadfield-Menell MU AAML ELM 122 6 0 03 Feb 2025
HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor Zihui Wu Haichang Gao Jiacheng Luo Zhaoxiang Liu 119 0 0 23 Jan 2025
ICLR: In-Context Learning of Representations Core Francisco Park Andrew Lee Ekdeep Singh Lubana Yongyi Yang Maya Okawa Kento Nishi Martin Wattenberg Hidenori Tanaka AIFin 210 6 0 29 Dec 2024
SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkage Xiaoning Dong Wenbo Hu Wei Xu Tianxing He 160 0 0 19 Dec 2024
Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models Javier Ferrando Oscar Obeso Senthooran Rajamanoharan Neel Nanda 153 29 0 21 Nov 2024
Controllable Context Sensitivity and the Knob Behind It Julian Minder Kevin Du Niklas Stoehr Giovanni Monea Chris Wendler Robert West Ryan Cotterell KELM 105 6 0 11 Nov 2024
All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling Emanuele Marconato Sébastien Lachapelle Sebastian Weichwald Luigi Gresele 102 4 0 30 Oct 2024
Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks Samuele Poppi Zheng-Xin Yong Yifei He Bobbie Chern Han Zhao Aobo Yang Jianfeng Chi AAML 132 20 0 23 Oct 2024
On the Role of Attention Heads in Large Language Model Safety Zhenhong Zhou Haiyang Yu Xinghua Zhang Rongwu Xu Fei Huang Kun Wang Yang Liu Sihang Li Yongbin Li 127 9 0 17 Oct 2024
Improving Instruction-Following in Language Models through Activation Steering Alessandro Stolfo Vidhisha Balachandran Safoora Yousefi Eric Horvitz Besmira Nushi LLMSV 115 26 0 15 Oct 2024
Towards Interpreting Visual Information Processing in Vision-Language Models Clement Neo Luke Ong Philip Torr Mor Geva David M. Krueger Fazl Barez 127 12 0 09 Oct 2024
Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation Xinpeng Wang Chengzhi Hu Paul Röttger Barbara Plank 128 10 0 04 Oct 2024
Robust LLM safeguarding via refusal feature adversarial training L. Yu Virginie Do Karen Hambardzumyan Nicola Cancedda AAML 115 16 0 30 Sep 2024
An Adversarial Perspective on Machine Unlearning for AI Safety Jakub Łucki Boyi Wei Yangsibo Huang Peter Henderson F. Tramèr Javier Rando MU AAML 138 46 0 26 Sep 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 149 32 0 02 Jul 2024
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks Maksym Andriushchenko Francesco Croce Nicolas Flammarion AAML 155 212 0 02 Apr 2024
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models Samuel Marks Can Rager Eric J. Michaud Yonatan Belinkov David Bau Aaron Mueller 133 151 0 28 Mar 2024
Yi: Open Foundation Models by 01.AI 01. AI Alex Young 01.AI Alex Young Bei Chen Chao Li ... Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai OSLM LRM 263 562 0 07 Mar 2024
A Language Model's Guide Through Latent Space Dimitri von Rutte Sotiris Anagnostidis Gregor Bachmann Thomas Hofmann 91 27 0 22 Feb 2024
Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methods Yotam Wolf Noam Wies Dorin Shteyman Binyamin Rothberg Yoav Levine Amnon Shashua LLMSV 106 13 0 29 Jan 2024
Steering Llama 2 via Contrastive Activation Addition Nina Rimsky Nick Gabrieli Julian Schulz Meg Tong Evan Hubinger Alexander Matt Turner LLMSV 57 211 0 09 Dec 2023
Removing RLHF Protections in GPT-4 via Fine-Tuning Qiusi Zhan Richard Fang R. Bindu Akul Gupta Tatsunori Hashimoto Daniel Kang MU AAML 56 101 0 09 Nov 2023
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks Alexander Robey Eric Wong Hamed Hassani George J. Pappas AAML 115 252 0 05 Oct 2023
LoFT: Local Proxy Fine-tuning For Improving Transferability Of Adversarial Attacks Against Large Language Model Muhammad Ahmed Shah Roshan S. Sharma Hira Dhamyal R. Olivier Ankit Shah ... Massa Baali Soham Deshmukh Michael Kuhlmann Bhiksha Raj Rita Singh AAML 41 21 0 02 Oct 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 262 1,827 0 28 Sep 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 291 1,455 0 27 Jul 2023
A Geometric Notion of Causal Probing Clément Guerner Anej Svete Tianyu Liu Alex Warstadt Ryan Cotterell LLMSV 90 17 0 27 Jul 2023
LEACE: Perfect linear concept erasure in closed form Nora Belrose David Schneider-Joseph Shauli Ravfogel Ryan Cotterell Edward Raff Stella Biderman KELM MU 81 118 0 06 Jun 2023
Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection P. Haghighatkhah Antske Fokkens Pia Sommerauer Bettina Speckmann Kevin Verbeek 62 14 0 08 Dec 2022
Discovering Latent Knowledge in Language Models Without Supervision Collin Burns Haotian Ye Dan Klein Jacob Steinhardt 131 375 0 07 Dec 2022
Toy Models of Superposition Nelson Elhage Tristan Hume Catherine Olsson Nicholas Schiefer T. Henighan ... Sam McCandlish Jared Kaplan Dario Amodei Martin Wattenberg C. Olah AAML MILM 183 368 0 21 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 874 12,973 0 04 Mar 2022
Red Teaming Language Models with Language Models Ethan Perez Saffron Huang Francis Song Trevor Cai Roman Ring John Aslanides Amelia Glaese Nat McAleese G. Irving AAML 166 664 0 07 Feb 2022
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 151 1,077 0 01 Nov 2021
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 285 4,408 0 27 Oct 2021