Composing Parameter-Efficient Modules with Arithmetic Operations

26 June 2023

Papers citing "Composing Parameter-Efficient Modules with Arithmetic Operations"

25 / 25 papers shown

Title
CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging Wenju Sun Qingyong Li Yangli-ao Geng Boyang Li MoMe 34 0 0 11 May 2025
Param $Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost Sheng Cao Mingrui Wu Karthik Prasad Yuandong Tian Zechun Liu MoMe 82 0 0 23 Apr 2025
SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearning Tianyang Xu Xiaoze Liu Feijie Wu Xiaoqian Wang Jing Gao MU 61 0 0 29 Mar 2025
MoMa: A Modular Deep Learning Framework for Material Property Prediction Botian Wang Y. Ouyang Yaohui Li Yansen Wang Haorui Cui Jianbing Zhang Xiaonan Wang Wei-Ying Ma Hao Zhou 49 0 0 21 Feb 2025
Superpose Singular Features for Model Merging Haiquan Qiu You Wu Quanming Yao MoMe 48 0 0 15 Feb 2025
Skill Expansion and Composition in Parameter Space Tenglong Liu J. Li Yinan Zheng Haoyi Niu Yixing Lan Xin Xu Xianyuan Zhan 58 4 0 09 Feb 2025
Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts Wenju Sun Qingyong Li Wen Wang Yangli-ao Geng Boyang Li 44 3 0 28 Jan 2025
IterIS: Iterative Inference-Solving Alignment for LoRA Merging Hongxu Chen Runshi Li Bowei Zhu Zhen Wang Long Chen MoMe 98 0 0 21 Nov 2024
CLEAR: Character Unlearning in Textual and Visual Modalities Alexey Dontsov Dmitrii Korzh Alexey Zhavoronkin Boris Mikheev Denis Bobkov Aibek Alanov Oleg Y. Rogov Ivan Oseledets Elena Tutubalina AILaw VLM MU 66 5 0 23 Oct 2024
Closed-form merging of parameter-efficient modules for Federated Continual Learning Riccardo Salami Pietro Buzzega Matteo Mosconi Jacopo Bonato Luigi Sabetta Simone Calderara FedML MoMe CLL 39 2 0 23 Oct 2024
Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace Jinluan Yang Anke Tang Didi Zhu Zhengyu Chen Li Shen Fei Wu MoMe AAML 62 3 0 17 Oct 2024
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 52 3 0 02 Oct 2024
MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearning Jiali Cheng Hadi Amiri BDL 43 3 0 21 Jun 2024
Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs S. Kadhe Farhan Ahmed Dennis Wei Nathalie Baracaldo Inkit Padhi MoMe MU 28 7 0 17 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 45 6 0 07 Jun 2024
Ensembling Diffusion Models via Adaptive Feature Aggregation Cong Wang Kuan Tian Yonghang Guan Jun Zhang Zhiwei Jiang Fei Shen Xiao Han 44 5 0 27 May 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su E. Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Caccia Alessandro Sordoni MoMe 44 31 0 18 May 2024
A safety realignment framework via subspace-oriented model fusion for large language models Xin Yi Shunfan Zheng Linlin Wang Xiaoling Wang Liang He 60 20 0 15 May 2024
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning Jinghan Jia Yihua Zhang Yimeng Zhang Jiancheng Liu Bharat Runwal James Diffenderfer B. Kailkhura Sijia Liu MU 43 35 0 28 Apr 2024
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization Alexandra Chronopoulou Jonas Pfeiffer Joshua Maynez Xinyi Wang Sebastian Ruder Priyanka Agrawal MoMe 26 16 0 15 Nov 2023
Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation Xinshuo Hu Dongfang Li Baotian Hu Zihao Zheng Zhenyu Liu Hao Fei KELM MU 33 26 0 16 Aug 2023
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 316 0 11 Sep 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 225 339 0 02 Feb 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,661 0 15 Oct 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,984 0 20 Apr 2018