MUSE: Machine Unlearning Six-Way Evaluation for Language Models

8 July 2024

Luke Zettlemoyer

Papers citing "MUSE: Machine Unlearning Six-Way Evaluation for Language Models"

43 / 43 papers shown

Title
WaterDrum: Watermarking for Data-centric Unlearning Metric Xinyang Lu Xinyuan Niu Gregory Kang Ruey Lau Bui Thi Cam Nhung Rachael Hwee Ling Sim Fanyu Wen Chuan-Sheng Foo S. Ng Bryan Kian Hsiang Low MU 59 0 0 08 May 2025
Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions Yiming Du Wenyu Huang Danna Zheng Zhaowei Wang Sébastien Montella Mirella Lapata Kam-Fai Wong Jeff Z. Pan KELM MU 78 2 0 01 May 2025
ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data Tong Chen Faeze Brahman Jiacheng Liu Niloofar Mireshghallah Weijia Shi Pang Wei Koh Luke Zettlemoyer Hannaneh Hajishirzi 40 0 0 20 Apr 2025
A mean teacher algorithm for unlearning of language models Yegor Klochkov MU 147 0 0 18 Apr 2025
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers Hongkang Li Yihua Zhang Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen MoMe 69 3 0 15 Apr 2025
$SAEs $\textit{Can}$ Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs$ SAEs $\textit{Can}$ Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs Aashiq Muhamed Jacopo Bonato Mona Diab Virginia Smith MU 61 0 0 11 Apr 2025
Understanding Machine Unlearning Through the Lens of Mode Connectivity Jiali Cheng Hadi Amiri MU 132 0 0 08 Apr 2025
Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outs Dongyang Fan Vinko Sabolčec Matin Ansaripour Ayush Kumar Tarun Martin Jaggi Antoine Bosselut Imanol Schlag 34 0 0 08 Apr 2025
Not All Data Are Unlearned Equally Aravind Krishnan Siva Reddy Marius Mosbach MU 142 1 0 07 Apr 2025
SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models Anil Ramakrishna Yixin Wan Xiaomeng Jin Kai-Wei Chang Zhiqi Bu Bhanukiran Vinzamuri V. Cevher Mingyi Hong Rahul Gupta AILaw MU 162 0 0 02 Apr 2025
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-tuning Yiwei Chen Yuguang Yao Yihua Zhang Bingquan Shen Gaowen Liu Sijia Liu AAML MU 58 1 0 14 Mar 2025
Erasing Without Remembering: Safeguarding Knowledge Forgetting in Large Language Models Huazheng Wang Yongcheng Jing Haifeng Sun Yingjie Wang J. Wang Jianxin Liao Dacheng Tao KELM MU 47 0 0 27 Feb 2025
FaithUn: Toward Faithful Forgetting in Language Models by Investigating the Interconnectedness of Knowledge Nakyeong Yang Minsung Kim Seunghyun Yoon Joongbo Shin Kyomin Jung KELM MU 65 0 0 26 Feb 2025
A General Framework to Enhance Fine-tuning-based LLM Unlearning J. Ren Zhenwei Dai X. Tang Hui Liu Jingying Zeng ... R. Goutam Suhang Wang Yue Xing Qi He Hui Liu MU 163 1 0 25 Feb 2025
Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models Haokun Chen Sebastian Szyller Weilin Xu N. Himayat MU AAML 43 0 0 20 Feb 2025
LUME: LLM Unlearning with Multitask Evaluations Anil Ramakrishna Yixin Wan Xiaomeng Jin Kai-Wei Chang Zhiqi Bu Bhanukiran Vinzamuri V. Cevher Mingyi Hong Rahul Gupta CLL MU 103 7 0 20 Feb 2025
Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models Yingqian Cui Pengfei He Jingying Zeng Hui Liu X. Tang ... Zhen Li Suhang Wang Yue Xing Jiliang Tang Qi He LRM 49 7 0 18 Feb 2025
ReLearn: Unlearning via Learning for Large Language Models Haoming Xu Ningyuan Zhao Liming Yang Sendong Zhao Shumin Deng Mengru Wang Bryan Hooi Nay Oo H. Chen N. Zhang KELM CLL MU 159 0 0 16 Feb 2025
Adversarial ML Problems Are Getting Harder to Solve and to Evaluate Javier Rando Jie Zhang Nicholas Carlini F. Tramèr AAML ELM 59 3 0 04 Feb 2025
Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classification Changchang Sun Ren Wang Yihua Zhang Jinghan Jia Jiancheng Liu Gaowen Liu Sijia Liu Yan Yan AAML MU 93 0 0 21 Dec 2024
Unlearning in- vs. out-of-distribution data in LLMs under gradient-based method Teodora Baluta Pascal Lamblin Daniel Tarlow Fabian Pedregosa Gintare Karolina Dziugaite MU 32 1 0 07 Nov 2024
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate Zhiqi Bu Xiaomeng Jin Bhanukiran Vinzamuri Anil Ramakrishna Kai-Wei Chang V. Cevher Mingyi Hong MU 85 6 0 29 Oct 2024
CLEAR: Character Unlearning in Textual and Visual Modalities Alexey Dontsov Dmitrii Korzh Alexey Zhavoronkin Boris Mikheev Denis Bobkov Aibek Alanov Oleg Y. Rogov Ivan V. Oseledets Elena Tutubalina AILaw VLM MU 66 5 0 23 Oct 2024
Catastrophic Failure of LLM Unlearning via Quantization Zhiwei Zhang Fali Wang Xiaomin Li Zongyu Wu Xianfeng Tang Hui Liu Qi He Wenpeng Yin Suhang Wang MU 34 5 0 21 Oct 2024
LLM Unlearning via Loss Adjustment with Only Forget Data Yaxuan Wang Jiaheng Wei Chris Liu Jinlong Pang Q. Liu A. Shah Yujia Bao Yang Liu Wei Wei KELM MU 37 7 0 14 Oct 2024
A Closer Look at Machine Unlearning for Large Language Models Xiaojian Yuan Tianyu Pang Chao Du Kejiang Chen Weiming Zhang Min-Bin Lin MU 41 5 0 10 Oct 2024
Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning Chongyu Fan Jiancheng Liu Licong Lin Jinghan Jia Ruiqi Zhang Song Mei Sijia Liu MU 43 16 0 09 Oct 2024
From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing Sarah H. Cen Rohan Alur 29 1 0 07 Oct 2024
Position: LLM Unlearning Benchmarks are Weak Measures of Progress Pratiksha Thaker Shengyuan Hu Neil Kale Yash Maurya Zhiwei Steven Wu Virginia Smith MU 53 10 0 03 Oct 2024
An Adversarial Perspective on Machine Unlearning for AI Safety Jakub Łucki Boyi Wei Yangsibo Huang Peter Henderson F. Tramèr Javier Rando MU AAML 71 32 0 26 Sep 2024
Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models Hongbang Yuan Zhuoran Jin Pengfei Cao Yubo Chen Kang Liu Jun Zhao AAML ELM MU 44 1 0 20 Aug 2024
The Future of Open Human Feedback Shachar Don-Yehiya Ben Burtenshaw Ramon Fernandez Astudillo Cailean Osborne Mimansa Jaiswal ... Omri Abend Jennifer Ding Sara Hooker Hannah Rose Kirk Leshem Choshen VLM ALM 62 4 0 15 Aug 2024
Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximization Zhaohan Zhang Ziquan Liu Ioannis Patras 36 2 0 09 Aug 2024
Evaluating Copyright Takedown Methods for Language Models Boyi Wei Weijia Shi Yangsibo Huang Noah A. Smith Chiyuan Zhang Luke Zettlemoyer Kai Li Peter Henderson 49 19 0 26 Jun 2024
Fantastic Copyrighted Beasts and How (Not) to Generate Them Luxi He Yangsibo Huang Weijia Shi Tinghao Xie Haotian Liu Yue Wang Luke Zettlemoyer Chiyuan Zhang Danqi Chen Peter Henderson 46 9 0 20 Jun 2024
Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning Ruiqi Zhang Licong Lin Yu Bai Song Mei MU 60 126 0 08 Apr 2024
Rethinking Machine Unlearning for Large Language Models Sijia Liu Yuanshun Yao Jinghan Jia Stephen Casper Nathalie Baracaldo ... Hang Li Kush R. Varshney Mohit Bansal Sanmi Koyejo Yang Liu AILaw MU 72 81 0 13 Feb 2024
Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications Boyi Wei Kaixuan Huang Yangsibo Huang Tinghao Xie Xiangyu Qi Mengzhou Xia Prateek Mittal Mengdi Wang Peter Henderson AAML 55 79 0 07 Feb 2024
Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges Nianwen Si Hao Zhang Heyu Chang Wenlin Zhang Dan Qu Weiqiang Zhang KELM MU 77 26 0 27 Nov 2023
Who's Harry Potter? Approximate Unlearning in LLMs Ronen Eldan M. Russinovich MU MoMe 101 173 0 03 Oct 2023
Knowledge Unlearning for Mitigating Privacy Risks in Language Models Joel Jang Dongkeun Yoon Sohee Yang Sungmin Cha Moontae Lee Lajanugen Logeswaran Minjoon Seo KELM PILM MU 147 190 0 04 Oct 2022
A Survey of Machine Unlearning Thanh Tam Nguyen T. T. Huynh Phi Le Nguyen Alan Wee-Chung Liew Hongzhi Yin Quoc Viet Hung Nguyen MU 77 221 0 06 Sep 2022
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,814 0 14 Dec 2020