v1v2 (latest)

An Empirical Study of Multimodal Model Merging

28 April 2023

ArXiv (abs)PDF HTML Github (38★)

Papers citing "An Empirical Study of Multimodal Model Merging"

33 / 33 papers shown

Title
Subspace-Boosted Model Merging Ronald Skorobogat Karsten Roth Mariana-Iuliana Georgescu Zeynep Akata MoMe 18 0 0 19 Jun 2025
Simplified Swarm Learning Framework for Robust and Scalable Diagnostic Services in Cancer Histopathology Yanjie Wu Yuhao Ji Saiho Lee Juniad Akram Ali Braytee Ali Anaissi 108 0 0 23 Apr 2025
AdaMMS: Model Merging for Heterogeneous Multimodal Large Language Models with Unsupervised Coefficient Optimization Yiyang Du Xiaochen Wang Cai Chen Jiabo Ye Yiru Wang ... J.N. Zhang Fei Huang Zhifang Sui Maosong Sun Yi Liu MoMe 95 1 0 31 Mar 2025
When Domain Generalization meets Generalized Category Discovery: An Adaptive Task-Arithmetic Driven Approach Vaibhav Rathore S. Bagchi Saikat Dutta Sarthak Mehrotra Zsolt Kira Biplab Banerjee OOD 108 1 0 19 Mar 2025
From Task-Specific Models to Unified Systems: A Review of Model Merging Approaches Wei Ruan Tianze Yang Yimiao Zhou Tianming Liu Jin Lu MoMe 128 0 0 13 Mar 2025
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation Fanhu Zeng Haiyang Guo Fei Zhu Li Shen Hao Tang MoMe 222 4 0 24 Feb 2025
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr VLM ObjD 545 1 0 12 Dec 2024
How to Merge Your Multimodal Models Over Time? Sebastian Dziadzio Vishaal Udandarao Karsten Roth Ameya Prabhu Zeynep Akata Samuel Albanie Matthias Bethge MoMe 187 7 0 09 Dec 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao N. Zhang Shumin Deng Ningyu Zhang MoMe 128 1 0 16 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Joey Tianyi Zhou Tsendsuren Munkhdalai MoMe 100 22 0 04 Oct 2024
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks Edan Kinderman Itay Hubara Haggai Maron Daniel Soudry MoMe 98 2 0 02 Oct 2024
Realistic Evaluation of Model Merging for Compositional Generalization Derek Tam Yash Kant Brian Lester Igor Gilitschenski Colin Raffel MoMe 97 9 0 26 Sep 2024
ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models Sombit Dey Jan-Nico Zaech Nikolay Nikolov Luc Van Gool Danda Pani Paudel MoMe VLM 127 5 0 23 Sep 2024
Rethinking Visual Prompting for Multimodal Large Language Models with External Knowledge Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr Lu Yuan LRM VLM 79 8 0 05 Jul 2024
Model Merging and Safety Alignment: One Bad Model Spoils the Bunch Hasan Hammoud Umberto Michieli Fabio Pizzati Philip Torr Adel Bibi Guohao Li Mete Ozay MoMe 71 18 0 20 Jun 2024
Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognition Hsuan Su Hua Farn Fan-Yun Sun Shang-Tse Chen Hung-yi Lee MoMe 75 5 0 05 Jun 2024
MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models Nithin Gopalakrishnan Nair Jeya Maria Jose Valanarasu Vishal M. Patel MoMe 83 7 0 15 Apr 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 78 2 0 11 Apr 2024
SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data Jialu Li Jaemin Cho Yi-Lin Sung Jaehong Yoon Mohit Bansal MoMe DiffM 99 9 0 11 Mar 2024
Model Composition for Multimodal Large Language Models Chi Chen Yiyang Du Zheng Fang Ziyue Wang Ziyue Wang ... Ming Yan Ji Zhang Fei Huang Maosong Sun Yang Liu MoMe 78 3 0 20 Feb 2024
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition Chen Chen Ruizhe Li Yuchen Hu Sabato Marco Siniscalchi Pin-Yu Chen Ensiong Chng Chao-Han Huck Yang 96 22 0 08 Feb 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Qing Cui Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 142 88 0 02 Jan 2024
Merging Vision Transformers from Different Tasks and Domains Peng Ye Chenyu Huang Mingzhu Shen Tao Chen Yongqi Huang Yuning Zhang Wanli Ouyang MoMe 77 12 0 25 Dec 2023
Multimodal Attention Merging for Improved Speech Recognition and Audio Event Classification Anirudh S. Sundar Chao-Han Huck Yang David M. Chan Shalini Ghosh Venkatesh Ravichandran P. S. Nidadavolu MoMe 94 9 0 22 Dec 2023
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 99 12 0 07 Dec 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 89 14 0 22 Nov 2023
Multimodal Representation Learning by Alternating Unimodal Adaptation Xiaohui Zhang Jaehong Yoon Mohit Bansal Huaxiu Yao 116 26 0 17 Nov 2023
SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding Haoxiang Wang Pavan Kumar Anasosalu Vasu Fartash Faghri Raviteja Vemulapalli Mehrdad Farajtabar Sachin Mehta Mohammad Rastegari Oncel Tuzel Hadi Pouransari VLM 128 72 0 23 Oct 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 113 62 0 27 Sep 2023
Unified Coarse-to-Fine Alignment for Video-Text Retrieval Ziyang Wang Yi-Lin Sung Feng Cheng Gedas Bertasius Joey Tianyi Zhou 208 49 0 18 Sep 2023
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks Mustafa Shukor Corentin Dancette Alexandre Ramé Matthieu Cord MoMe MLLM 122 46 0 30 Jul 2023
TIES-Merging: Resolving Interference When Merging Models Prateek Yadav Derek Tam Leshem Choshen Colin Raffel Joey Tianyi Zhou MoMe 141 318 0 02 Jun 2023
ZipIt! Merging Models from Different Tasks without Training George Stoica Daniel Bolya J. Bjorner Pratik Ramesh Taylor N. Hearn Judy Hoffman VLM MoMe 139 125 0 04 May 2023