Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning

9 October 2024

Jiancheng Liu

Sijia Liu

Papers citing "Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning"

50 / 64 papers shown

Title
Pre-training Large Memory Language Models with Internal and External Knowledge Linxi Zhao Sofian Zalouk Christian K. Belardi Justin Lovelace Jin Peng Zhou Kilian Q. Weinberger Yoav Artzi Jennifer J. Sun KELM HILM 58 0 0 21 May 2025
Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning Puning Yang Qizhou Wang Zhuo Huang Tongliang Liu Chengqi Zhang Bo Han MU 92 0 0 17 May 2025
Layered Unlearning for Adversarial Relearning Timothy Qian Vinith Suriyakumar Ashia Wilson Dylan Hadfield-Menell MU 63 1 0 14 May 2025
Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation Stefan Vasilev Christian Herold Baohao Liao Seyyed Hadi Hashemi Shahram Khadivi Christof Monz MU 383 0 0 09 May 2025
A mean teacher algorithm for unlearning of language models Yegor Klochkov MU 291 0 0 18 Apr 2025
$SAEs $\textit{Can}$ Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs$ SAEs $\textit{Can}$ Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs Aashiq Muhamed Jacopo Bonato Mona Diab Virginia Smith MU 105 4 0 11 Apr 2025
Bridging the Gap Between Preference Alignment and Machine Unlearning Xiaohua Feng Yuyuan Li Huwei Ji Jiaming Zhang Lulu Zhang Tianyu Du Chaochao Chen MU 57 0 0 09 Apr 2025
Understanding Machine Unlearning Through the Lens of Mode Connectivity Jiali Cheng Hadi Amiri MU 381 1 0 08 Apr 2025
Not All Data Are Unlearned Equally Aravind Krishnan Siva Reddy Marius Mosbach MU 300 1 0 07 Apr 2025
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-tuning Yiwei Chen Yuguang Yao Yihua Zhang Bingquan Shen Gaowen Liu Sijia Liu AAML MU 84 1 0 14 Mar 2025
Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models Huazheng Wang Yongcheng Jing Haifeng Sun Yingjie Wang Jingchao Wang Jianxin Liao Dacheng Tao KELM MU 83 0 0 27 Feb 2025
A General Framework to Enhance Fine-tuning-based LLM Unlearning J. Ren Zhenwei Dai Xianfeng Tang Hui Liu Jingying Zeng ... R. Goutam Suhang Wang Yue Xing Qi He Hui Liu MU 198 1 0 25 Feb 2025
MUSE: Machine Unlearning Six-Way Evaluation for Language Models Weijia Shi Jaechan Lee Yangsibo Huang Sadhika Malladi Jieyu Zhao Ari Holtzman Daogao Liu Luke Zettlemoyer Noah A. Smith Chiyuan Zhang MU ELM 70 66 0 08 Jul 2024
Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs S. Kadhe Farhan Ahmed Dennis Wei Nathalie Baracaldo Inkit Padhi MoMe MU 57 7 0 17 Jun 2024
Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition Eleni Triantafillou Peter Kairouz Fabian Pedregosa Jamie Hayes M. Kurmanji ... Lisheng Sun-Hosoya Sergio Escalera Gintare Karolina Dziugaite Peter Triantafillou Isabelle M Guyon MU 68 16 0 13 Jun 2024
Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference Jiabao Ji Yujian Liu Yang Zhang Gaowen Liu Ramana Rao Kompella Sijia Liu Shiyu Chang KELM MU 72 29 0 12 Jun 2024
Large Scale Knowledge Washing Yu Wang Ruihan Wu Zexue He Xinyu Chen Julian McAuley MU KELM 104 7 0 26 May 2024
SimPO: Simple Preference Optimization with a Reference-Free Reward Yu Meng Mengzhou Xia Danqi Chen 106 442 0 23 May 2024
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning Jinghan Jia Yihua Zhang Yimeng Zhang Jiancheng Liu Bharat Runwal James Diffenderfer B. Kailkhura Sijia Liu MU 73 41 0 28 Apr 2024
Rethinking LLM Memorization through the Lens of Adversarial Compression Avi Schwarzschild Zhili Feng Pratyush Maini Zachary Chase Lipton J. Zico Kolter 101 49 0 23 Apr 2024
Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning Ruiqi Zhang Licong Lin Yu Bai Song Mei MU 107 166 0 08 Apr 2024
Digital Forgetting in Large Language Models: A Survey of Unlearning Methods Alberto Blanco-Justicia N. Jebreel Benet Manzanares-Salor David Sánchez Josep Domingo-Ferrer Guillem Collell Kuan Eeik Tan KELM MU 83 20 0 02 Apr 2024
Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning Chongyu Fan Jiancheng Liu Alfred Hero Sijia Liu MU 54 31 0 12 Mar 2024
Guardrail Baselines for Unlearning in LLMs Pratiksha Thaker Yash Maurya Shengyuan Hu Zhiwei Steven Wu Virginia Smith MU 74 45 0 05 Mar 2024
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning Nathaniel Li Alexander Pan Anjali Gopal Summer Yue Daniel Berrios ... Yan Shoshitaishvili Jimmy Ba K. Esvelt Alexandr Wang Dan Hendrycks ELM 82 172 0 05 Mar 2024
Eight Methods to Evaluate Robust Unlearning in LLMs Aengus Lynch Phillip Guo Aidan Ewart Stephen Casper Dylan Hadfield-Menell ELM MU 84 71 0 26 Feb 2024
Rethinking Machine Unlearning for Large Language Models Sijia Liu Yuanshun Yao Jinghan Jia Stephen Casper Nathalie Baracaldo ... Hang Li Kush R. Varshney Mohit Bansal Sanmi Koyejo Yang Liu AILaw MU 108 100 0 13 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 238 527 0 02 Feb 2024
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models Xinwei Wu Junzhuo Li Minghui Xu Weilong Dong Shuangzhi Wu Chao Bian Deyi Xiong MU KELM 62 49 0 31 Oct 2023
Detecting Pretraining Data from Large Language Models Weijia Shi Anirudh Ajith Mengzhou Xia Yangsibo Huang Daogao Liu Terra Blevins Danqi Chen Luke Zettlemoyer MIALM 61 185 0 25 Oct 2023
SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation Chongyu Fan Jiancheng Liu Yihua Zhang Eric Wong Dennis Wei Sijia Liu MU 52 140 0 19 Oct 2023
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 163 615 0 18 Oct 2023
Large Language Model Unlearning Yuanshun Yao Xiaojun Xu Yang Liu MU 69 127 0 14 Oct 2023
In-Context Unlearning: Language Models as Few Shot Unlearners Martin Pawelczyk Seth Neel Himabindu Lakkaraju MU 61 117 0 11 Oct 2023
Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks Vaidehi Patil Peter Hase Joey Tianyi Zhou KELM AAML 92 104 0 29 Sep 2023
Knowledge Sanitization of Large Language Models Yoichi Ishibashi Hidetoshi Shimodaira KELM 97 20 0 21 Sep 2023
Efficient RLHF: Reducing the Memory Usage of PPO Michael Santacroce Yadong Lu Han Yu Yuan-Fang Li Yelong Shen 43 31 0 01 Sep 2023
Identifying and Mitigating the Security Risks of Generative AI Clark W. Barrett Bradley L Boyd Ellie Burzstein Nicholas Carlini Brad Chen ... Zulfikar Ramzan Khawaja Shams D. Song Ankur Taly Diyi Yang SILM 68 93 0 28 Aug 2023
Secrets of RLHF in Large Language Models Part I: PPO Rui Zheng Shihan Dou Songyang Gao Yuan Hua Wei Shen ... Hang Yan Tao Gui Qi Zhang Xipeng Qiu Xuanjing Huang ALM OffRL 75 168 0 11 Jul 2023
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models Wei Ping Weixin Chen Hengzhi Pei Chulin Xie Mintong Kang ... Zinan Lin Yuk-Kit Cheng Sanmi Koyejo D. Song Yue Liu 89 413 0 20 Jun 2023
Forgettable Federated Linear Learning with Certified Data Removal Rui Jin Minghui Chen Qiong Zhang Xiaoxiao Li MU 53 11 0 03 Jun 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 313 3,895 0 29 May 2023
SLiC-HF: Sequence Likelihood Calibration with Human Feedback Yao-Min Zhao Rishabh Joshi Tianqi Liu Misha Khalman Mohammad Saleh Peter J. Liu 59 294 0 17 May 2023
RRHF: Rank Responses to Align Language Models with Human Feedback without tears Zheng Yuan Hongyi Yuan Chuanqi Tan Wei Wang Songfang Huang Feiran Huang ALM 143 369 0 11 Apr 2023
Model Sparsity Can Simplify Machine Unlearning Jinghan Jia Jiancheng Liu Parikshit Ram Yuguang Yao Gaowen Liu Yang Liu Pranay Sharma Sijia Liu MU 68 119 0 11 Apr 2023
Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models Eric Zhang Kai Wang Xingqian Xu Zhangyang Wang Humphrey Shi DiffM 71 188 0 30 Mar 2023
Erasing Concepts from Diffusion Models Rohit Gandikota Joanna Materzyñska Jaden Fiotto-Kaufman David Bau DiffM 65 302 0 13 Mar 2023
Towards Unbounded Machine Unlearning M. Kurmanji Peter Triantafillou Jamie Hayes Eleni Triantafillou MU 54 139 0 20 Feb 2023
Privacy Adhering Machine Un-learning in NLP Vinayshekhar Bannihatti Kumar Rashmi Gangadharaiah Dan Roth AILaw MU 44 29 0 19 Dec 2022
Editing Models with Task Arithmetic Gabriel Ilharco Marco Tulio Ribeiro Mitchell Wortsman Suchin Gururangan Ludwig Schmidt Hannaneh Hajishirzi Ali Farhadi KELM MoMe MU 170 486 0 08 Dec 2022