Eight Methods to Evaluate Robust Unlearning in LLMs

26 February 2024

Dylan Hadfield-Menell

ELM

ArXiv PDF HTML

Papers citing "Eight Methods to Evaluate Robust Unlearning in LLMs"

32 / 32 papers shown

Title
Layered Unlearning for Adversarial Relearning Timothy Qian Vinith Suriyakumar Ashia Wilson Dylan Hadfield-Menell MU 28 0 0 14 May 2025
WaterDrum: Watermarking for Data-centric Unlearning Metric Xinyang Lu Xinyuan Niu Gregory Kang Ruey Lau Bui Thi Cam Nhung Rachael Hwee Ling Sim Fanyu Wen Chuan-Sheng Foo Szu Hui Ng Bryan Kian Hsiang Low MU 61 0 0 08 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Jen-tse Huang Joey Tianyi Zhou AAML MU 96 3 0 01 May 2025
Adaptively profiling models with task elicitation Davis Brown Prithvi Balehannina Helen Jin Shreya Havaldar Hamed Hassani Eric Wong ALM ELM 111 0 0 03 Mar 2025
A General Framework to Enhance Fine-tuning-based LLM Unlearning J. Ren Zhenwei Dai Xianfeng Tang Hui Liu Jingying Zeng ... R. Goutam Suhang Wang Yue Xing Qi He Hui Liu MU 163 1 0 25 Feb 2025
Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning Hwan Chang Hwanhee Lee MU 49 0 0 17 Feb 2025
Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities Zora Che Stephen Casper Robert Kirk Anirudh Satheesh Stewart Slocum ... Zikui Cai Bilal Chughtai Y. Gal Furong Huang Dylan Hadfield-Menell MU AAML ELM 85 3 0 03 Feb 2025
WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models Jinghan Jia Jiancheng Liu Yihua Zhang Parikshit Ram Nathalie Baracaldo Sijia Liu MU 40 2 0 23 Oct 2024
Do Unlearning Methods Remove Information from Language Model Weights? Aghyad Deeb Fabien Roger AAML MU 50 14 0 11 Oct 2024
Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning Chongyu Fan Jiancheng Liu Licong Lin Jinghan Jia Ruiqi Zhang Song Mei Sijia Liu MU 43 17 0 09 Oct 2024
A Probabilistic Perspective on Unlearning and Alignment for Large Language Models Yan Scholten Stephan Günnemann Leo Schwinn MU 63 6 0 04 Oct 2024
Position: LLM Unlearning Benchmarks are Weak Measures of Progress Pratiksha Thaker Shengyuan Hu Neil Kale Yash Maurya Zhiwei Steven Wu Virginia Smith MU 53 10 0 03 Oct 2024
Erasing Conceptual Knowledge from Language Models Rohit Gandikota Sheridan Feucht Samuel Marks David Bau KELM ELM MU 50 6 0 03 Oct 2024
An Adversarial Perspective on Machine Unlearning for AI Safety Jakub Łucki Boyi Wei Yangsibo Huang Peter Henderson F. Tramèr Javier Rando MU AAML 73 32 0 26 Sep 2024
Tamper-Resistant Safeguards for Open-Weight LLMs Rishub Tamirisa Bhrugu Bharathi Long Phan Andy Zhou Alice Gatti ... Andy Zou Dawn Song Bo Li Dan Hendrycks Mantas Mazeika AAML MU 53 42 0 01 Aug 2024
Composable Interventions for Language Models Arinbjorn Kolbeinsson Kyle O'Brien Tianjin Huang Shanghua Gao Shiwei Liu ... Anurag J. Vaidya Faisal Mahmood Marinka Zitnik Tianlong Chen Thomas Hartvigsen KELM MU 89 5 0 09 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 82 19 0 02 Jul 2024
AI Sandbagging: Language Models can Strategically Underperform on Evaluations Teun van der Weij Felix Hofstätter Ollie Jaffe Samuel F. Brown Francis Rhys Ward ELM 47 23 0 11 Jun 2024
Stress-Testing Capability Elicitation With Password-Locked Models Ryan Greenblatt Fabien Roger Dmitrii Krasheninnikov David M. Krueger 38 14 0 29 May 2024
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning Jinghan Jia Yihua Zhang Yimeng Zhang Jiancheng Liu Bharat Runwal James Diffenderfer B. Kailkhura Sijia Liu MU 43 35 0 28 Apr 2024
Threats, Attacks, and Defenses in Machine Unlearning: A Survey Ziyao Liu Huanyi Ye Chen Chen Yongsen Zheng K. Lam AAML MU 35 28 0 20 Mar 2024
Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications Boyi Wei Kaixuan Huang Yangsibo Huang Tinghao Xie Xiangyu Qi Mengzhou Xia Prateek Mittal Mengdi Wang Peter Henderson AAML 63 79 0 07 Feb 2024
A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity Andrew Lee Xiaoyan Bai Itamar Pres Martin Wattenberg Jonathan K. Kummerfeld Rada Mihalcea 77 96 0 03 Jan 2024
Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness? Kevin Liu Stephen Casper Dylan Hadfield-Menell Jacob Andreas HILM 64 36 0 27 Nov 2023
Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges Nianwen Si Hao Zhang Heyu Chang Wenlin Zhang Dan Qu Weiqiang Zhang KELM MU 80 26 0 27 Nov 2023
Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks Erfan Shayegani Md Abdullah Al Mamun Yu Fu Pedram Zaree Yue Dong Nael B. Abu-Ghazaleh AAML 147 148 0 16 Oct 2023
Who's Harry Potter? Approximate Unlearning in LLMs Ronen Eldan M. Russinovich MU MoMe 112 176 0 03 Oct 2023
Knowledge Unlearning for Mitigating Privacy Risks in Language Models Joel Jang Dongkeun Yoon Sohee Yang Sungmin Cha Moontae Lee Lajanugen Logeswaran Minjoon Seo KELM PILM MU 147 193 0 04 Oct 2022
A Survey of Machine Unlearning Thanh Tam Nguyen T. T. Huynh Phi Le Nguyen Alan Wee-Chung Liew Hongzhi Yin Quoc Viet Hung Nguyen MU 79 222 0 06 Sep 2022
Linear Connectivity Reveals Generalization Strategies Jeevesh Juneja Rachit Bansal Kyunghyun Cho João Sedoc Naomi Saphra 244 45 0 24 May 2022
Fine-tuned Language Models are Continual Learners Thomas Scialom Tuhin Chakrabarty Smaranda Muresan CLL LRM 145 117 0 24 May 2022
Probing Classifiers: Promises, Shortcomings, and Advances Yonatan Belinkov 226 405 0 24 Feb 2021