TIES-Merging: Resolving Interference When Merging Models

2 June 2023

Papers citing "TIES-Merging: Resolving Interference When Merging Models"

50 / 221 papers shown

Title
Collective Model Intelligence Requires Compatible Specialization Jyothish Pari Samy Jelassi Pulkit Agrawal MoMe 51 1 0 04 Nov 2024
Learning and Unlearning of Fabricated Knowledge in Language Models Chen Sun Nolan Miller A. Zhmoginov Max Vladymyrov Mark Sandler KELM MU 35 1 0 29 Oct 2024
FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion Zhenheng Tang Yonggang Zhang Peijie Dong Y. Cheung Amelie Chi Zhou Bo Han Xiaowen Chu FedML MoMe AI4CE 53 6 0 27 Oct 2024
Model merging with SVD to tie the Knots George Stoica Pratik Ramesh B. Ecsedi Leshem Choshen Judy Hoffman MoMe 39 9 0 25 Oct 2024
Inference time LLM alignment in single and multidomain preference spectrum Shri Kiran Srinivasan Zheng Qi Nikolaos Pappas Srikanth Doss Kadarundalagi Raghuram Doss Monica Sunkara Kishaloy Halder Manuel Mager Yassine Benajiba 37 0 0 24 Oct 2024
Closed-form merging of parameter-efficient modules for Federated Continual Learning Riccardo Salami Pietro Buzzega Matteo Mosconi Jacopo Bonato Luigi Sabetta Simone Calderara FedML MoMe CLL 39 2 0 23 Oct 2024
Can Large Language Models Invent Algorithms to Improve Themselves? Yoichi Ishibashi Taro Yano Masafumi Oyamada AIFin LRM 39 1 0 21 Oct 2024
Collaboratively adding new knowledge to an LLM Rhui Dih Lee L. Wynter CLL MoMe 32 0 0 18 Oct 2024
A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models Qiaoyu Tang Le Yu Bowen Yu Hongyu Lin Keming Lu Yaojie Lu Xianpei Han Le Sun MoMe 34 1 0 17 Oct 2024
Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace Jinluan Yang Anke Tang Didi Zhu Zhengyu Chen Li Shen Fei Wu MoMe AAML 62 3 0 17 Oct 2024
LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks Akshara Prabhakar Yuanzhi Li Karthik R. Narasimhan Sham Kakade Eran Malach Samy Jelassi MoMe 36 9 0 16 Oct 2024
The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse Ekansh Sharma Daniel M. Roy Gintare Karolina Dziugaite MoMe 42 2 0 16 Oct 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao N. Zhang Shumin Deng H. Chen MoMe 43 1 0 16 Oct 2024
Multi-trait User Simulation with Adaptive Decoding for Conversational Task Assistants Rafael Ferreira David Semedo João Magalhães 31 1 0 16 Oct 2024
Agent Skill Acquisition for Large Language Models via CycleQD So Kuroki Taishi Nakamura Takuya Akiba Yujin Tang MoMe 36 0 0 16 Oct 2024
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence Shangbin Feng Zifeng Wang Yike Wang Sayna Ebrahimi Hamid Palangi ... Nathalie Rauschmayr Yejin Choi Yulia Tsvetkov Chen-Yu Lee Tomas Pfister MoMe 39 3 0 15 Oct 2024
Self-Data Distillation for Recovering Quality in Pruned Large Language Models Vithursan Thangarasa Ganesh Venkatesh Mike Lasby Nish Sinnadurai Sean Lie SyDa 38 1 0 13 Oct 2024
CollabEdit: Towards Non-destructive Collaborative Knowledge Editing Jiamu Zheng Jinghuai Zhang Tianyu Du Xuhong Zhang Jianwei Yin Tao Lin KELM 40 0 0 12 Oct 2024
MergePrint: Merge-Resistant Fingerprints for Robust Black-box Ownership Verification of Large Language Models Shojiro Yamabe Tsubasa Takahashi Futa Waseda Koki Wataoka MoMe 86 1 0 11 Oct 2024
How Does Vision-Language Adaptation Impact the Safety of Vision Language Models? Seongyun Lee Geewook Kim Jiyeon Kim Hyunji Lee Hoyeon Chang Sue Hyun Park Minjoon Seo 36 0 0 10 Oct 2024
Extracting and Transferring Abilities For Building Multi-lingual Ability-enhanced Large Language Models Zhipeng Chen Liang Song K. Zhou Wayne Xin Zhao Binghui Wang Weipeng Chen Ji-Rong Wen 68 0 0 10 Oct 2024
Glider: Global and Local Instruction-Driven Expert Router Pingzhi Li Prateek Yadav Jaehong Yoon Jie Peng Yi-Lin Sung Joey Tianyi Zhou Tianlong Chen MoMe MoE 33 1 0 09 Oct 2024
Decouple-Then-Merge: Finetune Diffusion Models as Multi-Task Learning Qianli Ma Xuefei Ning Dongrui Liu Li Niu Linfeng Zhang MoMe 57 0 0 09 Oct 2024
NegMerge: Consensual Weight Negation for Strong Machine Unlearning Hyoseo Kim Dongyoon Han Junsuk Choe MoMe MU 36 1 0 08 Oct 2024
Low-Rank Continual Personalization of Diffusion Models Łukasz Staniszewski Katarzyna Zaleska Kamil Deja DiffM 44 0 0 07 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Joey Tianyi Zhou Tsendsuren Munkhdalai MoMe 46 16 0 04 Oct 2024
Parameter Competition Balancing for Model Merging Guodong Du Junlin Lee Jing Li Runhua Jiang Yifei Guo ... Hanting Liu S. Goh Ho-Kin Tang Daojing He Min Zhang MoMe 37 12 0 03 Oct 2024
DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation Changdae Oh Yixuan Li Kyungwoo Song Sangdoo Yun Dongyoon Han OOD MoMe 45 4 0 03 Oct 2024
Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging Tingfeng Hui Zhenyu Zhang Shuohuan Wang Yu Sun Hua Wu Sen Su MoE 31 0 0 02 Oct 2024
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks Edan Kinderman Itay Hubara Haggai Maron Daniel Soudry MoMe 52 1 0 02 Oct 2024
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 52 3 0 02 Oct 2024
The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging Masanori Hirano Kentaro Imajo MoMe 34 1 0 30 Sep 2024
HM3: Heterogeneous Multi-Class Model Merging Stefan Hackmann MoMe 30 0 0 27 Sep 2024
HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models Yu Zhou Xingyu Wu Jibin Wu Liang Feng Kay Chen Tan MoMe 61 0 0 27 Sep 2024
Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integration Mahdi Morafah Vyacheslav Kungurtsev Hojin Chang Chong Chen Bill Lin FedML 39 0 0 27 Sep 2024
CRoP: Context-wise Robust Static Human-Sensing Personalization Sawinder Kaur Avery Gump Yi Xiao Jingyu Xin Harshit Sharma Nina R Benway Jonathan L Preston Asif Salekin 29 0 0 26 Sep 2024
Prompt Sliders for Fine-Grained Control, Editing and Erasing of Concepts in Diffusion Models Deepak Sridhar Nuno Vasconcelos DiffM 36 0 0 25 Sep 2024
Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering Ziyu Zhao Tao Shen Didi Zhu Zexi Li Jing Su Xuwu Wang Kun Kuang Fei Wu MoMe 39 7 0 24 Sep 2024
Recent Advances in Attack and Defense Approaches of Large Language Models Jing Cui Yishi Xu Zhewei Huang Shuchang Zhou Jianbin Jiao Junge Zhang PILM AAML 57 1 0 05 Sep 2024
Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models Yuncheng Yang Yulei Qin Tong Wu Zihan Xu Gang Li ... Yuchen Shi Ke Li Xing Sun Jie Yang Yun Gu ALM OffRL MoE 60 0 0 28 Aug 2024
SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging Mohammadreza Pourreza Ruoxi Sun Hailong Li Lesly Miculicich Tomas Pfister Sercan Ö. Arik MoMe 40 5 0 22 Aug 2024
MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair Meghdad Dehghan Jie JW Wu Fatemeh H. Fard Ali Ouni MoMe 50 2 0 18 Aug 2024
Activated Parameter Locating via Causal Intervention for Model Merging Fanshuang Kong Richong Zhang Ziqiao Wang MoMe 21 1 0 18 Aug 2024
Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models Vladimir Araujo Marie-Francine Moens Tinne Tuytelaars CLL MoMe 28 2 0 16 Aug 2024
FuseChat: Knowledge Fusion of Chat Models Fanqi Wan Longguang Zhong Ziyi Yang Ruijun Chen Xiaojun Quan ALM KELM MoMe 32 23 0 15 Aug 2024
BadMerging: Backdoor Attacks Against Model Merging Jinghuai Zhang Jianfeng Chi Zheng Li Kunlin Cai Yang Zhang Yuan Tian MoMe FedML AAML 44 14 0 14 Aug 2024
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning Prateek Yadav Colin Raffel Mohammed Muqeeth Lucas Caccia Haokun Liu Tianlong Chen Joey Tianyi Zhou Leshem Choshen Alessandro Sordoni MoMe 49 21 0 13 Aug 2024
Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement Le Yu Bowen Yu Haiyang Yu Fei Huang Yongbin Li MoMe 35 5 0 06 Aug 2024
Efficient Pareto Manifold Learning with Low-Rank Structure Weiyu Chen James T. Kwok 36 7 0 30 Jul 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 80 1 0 30 Jul 2024