Deep Model Fusion: A Survey

27 September 2023

Liang Ding

Li Shen

Papers citing "Deep Model Fusion: A Survey"

50 / 53 papers shown

Title
Mitigating Parameter Interference in Model Merging via Sharpness-Aware Fine-Tuning Yeoreum Lee Jinwook Jung Sungyong Baik MoMe 40 0 0 20 Apr 2025
LeForecast: Enterprise Hybrid Forecast by Time Series Intelligence Zheng Tan Yiwen Nie Wenfa Wu Guanyu Zhang Yanze Liu ... Chao Yang Jiaxuan Fan Yuan He Hongsheng Qi Yangzhou Du AI4TS 42 0 0 27 Mar 2025
Union of Experts: Adapting Hierarchical Routing to Equivalently Decomposed Transformer Yujiao Yang Jing Lian Linhui Li MoE 77 0 0 04 Mar 2025
CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Merging Zongzhen Yang Binhang Qi Hailong Sun Wenrui Long Ruobing Zhao Xiang Gao MoMe 48 0 0 26 Feb 2025
R-MTLLMF: Resilient Multi-Task Large Language Model Fusion at the Wireless Edge Aladin Djuhera Vlad-Costin Andrei Mohsen Pourghasemian Haris Gacanin Holger Boche Walid Saad MoMe 113 0 0 24 Feb 2025
Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion Binchi Zhang Zaiyi Zheng Zhengzhang Chen Wenlin Yao 59 0 0 01 Feb 2025
Parameter-Efficient Interventions for Enhanced Model Merging Marcin Osial Daniel Marczak Bartosz Zieliñski MoMe 84 1 0 22 Dec 2024
Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learning Minghui Chen Meirui Jiang Xin Zhang Qi Dou Zehua Wang Xiaoxiao Li MoMe FedML 45 2 0 31 Oct 2024
SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery Enneng Yang Li Shen Zhenyi Wang G. Guo Xingwei Wang Xiaocun Cao Jie Zhang Dacheng Tao MoMe 37 4 0 18 Oct 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao N. Zhang Shumin Deng H. Chen MoMe 39 1 0 16 Oct 2024
Wolf2Pack: The AutoFusion Framework for Dynamic Parameter Fusion Bowen Tian Songning Lai Yutao Yue MoMe 30 0 0 08 Oct 2024
Parameter Competition Balancing for Model Merging Guodong Du Junlin Lee Jing Li Runhua Jiang Yifei Guo ... Hanting Liu S. Goh Ho-Kin Tang Daojing He Min Zhang MoMe 35 12 0 03 Oct 2024
House of Cards: Massive Weights in LLMs Jaehoon Oh Seungjun Shin Dokwan Oh 35 1 0 02 Oct 2024
The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging Masanori Hirano Kentaro Imajo MoMe 29 1 0 30 Sep 2024
HM3: Heterogeneous Multi-Class Model Merging Stefan Hackmann MoMe 30 0 0 27 Sep 2024
Layer-wise Model Merging for Unsupervised Domain Adaptation in Segmentation Tasks Roberto Alcover-Couso Juan C. Sanmiguel Marcos Escudero-Viñolo Jose M. Martínez FedML MoMe 28 1 0 24 Sep 2024
SwiftBrush v2: Make Your One-step Diffusion Model Better Than Its Teacher T. Dao Thuan Hoang Nguyen T. Le D. Vu Khoi Nguyen Cuong Pham Anh Tran DiffM 41 11 0 26 Aug 2024
Improving the Classification Effect of Clinical Images of Diseases for Multi-Source Privacy Protection Tian Bowen Xu Zhengyang Yin Zhihao Wang Jingying Yue Yutao FedML 37 0 0 23 Aug 2024
Weight Scope Alignment: A Frustratingly Easy Method for Model Merging Yichu Xu Xin-Chun Li Le Gan De-Chuan Zhan MoMe 40 0 0 22 Aug 2024
Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion Anke Tang Li Shen Yong Luo Shiwei Liu Han Hu Bo Du MoMe 28 6 0 14 Jun 2024
FusionBench: A Comprehensive Benchmark of Deep Model Fusion Anke Tang Li Shen Yong Luo Han Hu Bo Du Dacheng Tao ELM MoMe VLM 44 21 0 05 Jun 2024
Fusion-PSRO: Nash Policy Fusion for Policy Space Response Oracles Jiesong Lian Yucong Huang Chengdong Ma Mingzhi Wang Ying Wen Long Hu Yixue Hao 57 0 0 31 May 2024
WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models Peng Wang Zexi Li Ningyu Zhang Ziwen Xu Yunzhi Yao Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen KELM CLL 47 20 0 23 May 2024
Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks Xin-Chun Li Jinli Tang Bo Zhang Lan Li De-Chuan Zhan 46 2 0 21 May 2024
Post-Hoc Reversal: Are We Selecting Models Prematurely? Rishabh Ranjan Saurabh Garg Mrigank Raman Carlos Guestrin Zachary Chase Lipton 42 0 0 11 Apr 2024
MedMerge: Merging Models for Effective Transfer Learning to Medical Imaging Tasks Ibrahim Almakky Santosh Sanjeev Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub Mohammad Yaqub FedML MoMe 82 3 0 18 Mar 2024
Training-Free Pretrained Model Merging Zhenxing Xu Ke Yuan Huiqiong Wang Yong Wang Mingli Song Jie Song MoMe 32 15 0 04 Mar 2024
Representation Surgery for Multi-Task Model Merging Enneng Yang Li Shen Zhenyi Wang Guibing Guo Xiaojun Chen Xingwei Wang Dacheng Tao MoMe 56 37 0 05 Feb 2024
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts Anke Tang Li Shen Yong Luo Nan Yin Lefei Zhang Dacheng Tao MoMe 33 40 0 01 Feb 2024
FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models Feihong He Gang Li Mengyuan Zhang Leilei Yan Hui Xiong Fanzhang Li Li Shen DiffM 23 15 0 28 Jan 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Qing Cui Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 47 76 0 02 Jan 2024
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion Anke Tang Li Shen Yong Luo Liang Ding Han Hu Bo Du Dacheng Tao MoMe 33 21 0 11 Dec 2023
Fusing Multiple Algorithms for Heterogeneous Online Learning D. Gadginmath Shivanshu Tripathi Fabio Pasqualetti FedML 16 1 0 09 Dec 2023
Applications of Spiking Neural Networks in Visual Place Recognition S. Hussaini Michael Milford Tobias Fischer 66 6 0 22 Nov 2023
Merging Experts into One: Improving Computational Efficiency of Mixture of Experts Shwai He Run-Ze Fan Liang Ding Li Shen Dinesh Manocha Dacheng Tao MoE MoMe 32 14 0 15 Oct 2023
Parameter Efficient Multi-task Model Fusion with Partial Linearization Anke Tang Li Shen Yong Luo Yibing Zhan Han Hu Bo Du Yixin Chen Dacheng Tao MoMe 26 30 0 07 Oct 2023
AdaMerging: Adaptive Model Merging for Multi-Task Learning Enneng Yang Zhenyi Wang Li Shen Shiwei Liu Guibing Guo Xingwei Wang Dacheng Tao MoMe 35 97 0 04 Oct 2023
TIES-Merging: Resolving Interference When Merging Models Prateek Yadav Derek Tam Leshem Choshen Colin Raffel Joey Tianyi Zhou MoMe 40 253 0 02 Jun 2023
PopulAtion Parameter Averaging (PAPA) Alexia Jolicoeur-Martineau Emy Gervais Kilian Fatras Yan Zhang Simon Lacoste-Julien MoMe 40 17 0 06 Apr 2023
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 252 314 0 11 Sep 2022
Trainable Weight Averaging: Accelerating Training and Improving Generalization Tao Li Zhehao Huang Yingwen Wu Zhengbao He Qinghua Tao X. Huang Chih-Jen Lin MoMe 50 3 0 26 May 2022
Linear Connectivity Reveals Generalization Strategies Jeevesh Juneja Rachit Bansal Kyunghyun Cho João Sedoc Naomi Saphra 242 45 0 24 May 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 196 128 0 19 May 2022
Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry Fabrizio Pittorino Antonio Ferraro Gabriele Perugini Christoph Feinauer Carlo Baldassi R. Zecchina 201 24 0 07 Feb 2022
Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs Kaichao You Yong Liu Ziyang Zhang Jianmin Wang Michael I. Jordan Mingsheng Long 110 30 0 20 Oct 2021
Efficiently Identifying Task Groupings for Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 206 238 1 10 Sep 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 317 5,785 0 29 Apr 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 284 1,524 0 27 Feb 2021
SWAD: Domain Generalization by Seeking Flat Minima Junbum Cha Sanghyuk Chun Kyungjae Lee Han-Cheol Cho Seunghyun Park Yunsung Lee Sungrae Park MoMe 216 423 0 17 Feb 2021
Optimizing Mode Connectivity via Neuron Alignment N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 223 80 0 05 Sep 2020