Editing Models with Task Arithmetic

8 December 2022

Papers citing "Editing Models with Task Arithmetic"

50 / 355 papers shown

Title
SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data Jialu Li Jaemin Cho Yi-Lin Sung Jaehong Yoon Mohit Bansal MoMe DiffM 47 8 0 11 Mar 2024
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning Nathaniel Li Alexander Pan Anjali Gopal Summer Yue Daniel Berrios ... Yan Shoshitaishvili Jimmy Ba K. Esvelt Alexandr Wang Dan Hendrycks ELM 45 144 0 05 Mar 2024
Training-Free Pretrained Model Merging Zhenxing Xu Ke Yuan Huiqiong Wang Yong Wang Mingli Song Jie Song MoMe 32 15 0 04 Mar 2024
Dissecting Language Models: Machine Unlearning via Selective Pruning Nicholas Pochinkov Nandi Schoots MILM MU 20 15 0 02 Mar 2024
Eight Methods to Evaluate Robust Unlearning in LLMs Aengus Lynch Phillip Guo Aidan Ewart Stephen Casper Dylan Hadfield-Menell ELM MU 40 57 0 26 Feb 2024
Training Neural Networks from Scratch with Parallel Low-Rank Adapters Minyoung Huh Brian Cheung Jeremy Bernstein Phillip Isola Pulkit Agrawal 35 10 0 26 Feb 2024
InstructEdit: Instruction-based Knowledge Editing for Large Language Models Ningyu Zhang Bo Tian Siyuan Cheng Xiaozhuan Liang Yi Hu Kouying Xue Yanjie Gou Xi Chen Huajun Chen KELM 48 4 0 25 Feb 2024
Knowledge Fusion of Chat LLMs: A Preliminary Technical Report Fanqi Wan Ziyi Yang Longguang Zhong Xiaojun Quan Xinting Huang Wei Bi MoMe 27 1 0 25 Feb 2024
Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy? Nader Asadi Mahdi Beitollahi Yasser H. Khalil Yinchuan Li Guojun Zhang Xi Chen MoMe 40 8 0 23 Feb 2024
Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking Nikhil Prakash Tamar Rott Shaham Tal Haklay Yonatan Belinkov David Bau 49 52 0 22 Feb 2024
Q-Probe: A Lightweight Approach to Reward Maximization for Language Models Kenneth Li Samy Jelassi Hugh Zhang Sham Kakade Martin Wattenberg David Brandfonbrener 32 9 0 22 Feb 2024
Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization James Oldfield Markos Georgopoulos Grigorios G. Chrysos Christos Tzelepis Yannis Panagakis M. Nicolaou Jiankang Deng Ioannis Patras MoE 45 8 0 19 Feb 2024
Rethinking Machine Unlearning for Large Language Models Sijia Liu Yuanshun Yao Jinghan Jia Stephen Casper Nathalie Baracaldo ... Hang Li Kush R. Varshney Mohit Bansal Sanmi Koyejo Yang Liu AILaw MU 72 83 0 13 Feb 2024
Learning to Route Among Specialized Experts for Zero-Shot Generalization Mohammed Muqeeth Haokun Liu Yufan Liu Colin Raffel MoMe 37 34 0 08 Feb 2024
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm Zhanpeng Zhou Zijun Chen Yilan Chen Bo-Wen Zhang Junchi Yan MoMe 19 9 0 06 Feb 2024
Representation Surgery for Multi-Task Model Merging Enneng Yang Li Shen Zhenyi Wang Guibing Guo Xiaojun Chen Xingwei Wang Dacheng Tao MoMe 56 38 0 05 Feb 2024
MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers Yatong Bai Mo Zhou Vishal M. Patel Somayeh Sojoudi AAML 24 6 0 03 Feb 2024
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts Anke Tang Li Shen Yong Luo Nan Yin Lefei Zhang Dacheng Tao MoMe 36 41 0 01 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 37 4 0 31 Jan 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 116 93 0 22 Jan 2024
LLM Augmented LLMs: Expanding Capabilities through Composition Rachit Bansal Bidisha Samanta Siddharth Dalmia Nitish Gupta Shikhar Vashishth Sriram Ganapathy Abhishek Bapna Prateek Jain Partha P. Talukdar CLL 21 34 0 04 Jan 2024
PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning Haiyang Guo Fei Zhu Wenzhuo Liu Xu-Yao Zhang Cheng-Lin Liu CLL 32 6 0 04 Jan 2024
A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity Andrew Lee Xiaoyan Bai Itamar Pres Martin Wattenberg Jonathan K. Kummerfeld Rada Mihalcea 74 96 0 03 Jan 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Qing Cui Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 47 76 0 02 Jan 2024
Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformers Peng Ye Yongqi Huang Chongjun Tu Minglei Li Tao Chen Tong He Wanli Ouyang 30 4 0 25 Dec 2023
Merging Vision Transformers from Different Tasks and Domains Peng Ye Chenyu Huang Mingzhu Shen Tao Chen Yongqi Huang Yuning Zhang Wanli Ouyang MoMe 22 11 0 25 Dec 2023
Multimodal Attention Merging for Improved Speech Recognition and Audio Event Classification Anirudh S. Sundar Chao-Han Huck Yang David M. Chan Shalini Ghosh Venkatesh Ravichandran P. S. Nidadavolu MoMe 40 8 0 22 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 49 135 0 19 Dec 2023
Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Masks Mohammad-Javad Davari Eugene Belilovsky MoMe 40 56 0 11 Dec 2023
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion Anke Tang Li Shen Yong Luo Liang Ding Han Hu Bo Du Dacheng Tao MoMe 35 21 0 11 Dec 2023
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 21 10 0 07 Dec 2023
Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges Nianwen Si Hao Zhang Heyu Chang Wenlin Zhang Dan Qu Weiqiang Zhang KELM MU 80 26 0 27 Nov 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 32 13 0 22 Nov 2023
In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering Sheng Liu Haotian Ye Lei Xing James Y. Zou 26 84 0 11 Nov 2023
LCM-LoRA: A Universal Stable-Diffusion Acceleration Module Simian Luo Yiqin Tan Suraj Patil Daniel Gu Patrick von Platen Apolinário Passos Longbo Huang Jian Li Hang Zhao MoMe 110 144 0 09 Nov 2023
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch Le Yu Yu Bowen Haiyang Yu Fei Huang Yongbin Li MoMe 28 272 0 06 Nov 2023
A Survey on Knowledge Editing of Neural Networks Vittorio Mazzia Alessandro Pedrani Andrea Caciolai Kay Rottmann Davide Bernardi KELM 20 24 0 30 Oct 2023
SoK: Memorization in General-Purpose Large Language Models Valentin Hartmann Anshuman Suri Vincent Bindschaedler David E. Evans Shruti Tople Robert West KELM LLMAG 18 20 0 24 Oct 2023
SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding Haoxiang Wang Pavan Kumar Anasosalu Vasu Fartash Faghri Raviteja Vemulapalli Mehrdad Farajtabar Sachin Mehta Mohammad Rastegari Oncel Tuzel Hadi Pouransari VLM 32 67 0 23 Oct 2023
Function Vectors in Large Language Models Eric Todd Millicent Li Arnab Sen Sharma Aaron Mueller Byron C. Wallace David Bau 14 100 0 23 Oct 2023
Equivariant Deep Weight Space Alignment Aviv Navon Aviv Shamsian Ethan Fetaya Gal Chechik Nadav Dym Haggai Maron 29 21 0 20 Oct 2023
Model Merging by Uncertainty-Based Gradient Matching Nico Daheim Thomas Möllenhoff E. Ponti Iryna Gurevych Mohammad Emtiyaz Khan MoMe FedML 32 43 0 19 Oct 2023
Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging Joel Jang Seungone Kim Bill Yuchen Lin Yizhong Wang Jack Hessel Luke Zettlemoyer Hannaneh Hajishirzi Yejin Choi Prithviraj Ammanabrolu MoMe 48 132 0 17 Oct 2023
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective Ming Zhong Chenxin An Weizhu Chen Jiawei Han Pengcheng He 26 9 0 17 Oct 2023
Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting Melanie Sclar Yejin Choi Yulia Tsvetkov Alane Suhr 33 304 0 17 Oct 2023
Can We Edit Multimodal Large Language Models? Siyuan Cheng Bo Tian Qingbin Liu Xi Chen Yongheng Wang Huajun Chen Ningyu Zhang MLLM 28 28 0 12 Oct 2023
Measuring Feature Sparsity in Language Models Mingyang Deng Lucas Tao Joe Benton 21 1 0 11 Oct 2023
A Meta-Learning Perspective on Transformers for Causal Language Modeling Xinbo Wu L. Varshney 29 6 0 09 Oct 2023
Establishing Trustworthiness: Rethinking Tasks and Model Evaluation Robert Litschko Max Müller-Eberstein Rob van der Goot Leon Weber Barbara Plank LRM 21 2 0 09 Oct 2023
Uncovering hidden geometry in Transformers via disentangling position and context Jiajun Song Yiqiao Zhong 32 10 0 07 Oct 2023