Efficient and Effective Weight-Ensembling Mixture of Experts for
Multi-Task Model Merging

Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging

29 October 2024

Lefei Zhang

Dacheng Tao

Papers citing "Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging"

5 / 5 papers shown

Title
QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration HamidReza Imani Jiaxin Peng Peiman Mohseni Abdolah Amirany Tarek A. El-Ghazawi MoE 31 0 0 10 May 2025
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications Siyuan Mu Sen Lin MoE 135 2 0 10 Mar 2025
How to Merge Your Multimodal Models Over Time? Sebastian Dziadzio Vishaal Udandarao Karsten Roth Ameya Prabhu Zeynep Akata Samuel Albanie Matthias Bethge MoMe 98 3 0 09 Dec 2024
Task Singular Vectors: Reducing Task Interference in Model Merging Antonio Andrea Gargiulo Donato Crisostomi Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Emanuele Rodolà MoMe 87 9 0 26 Nov 2024
SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery Enneng Yang Li Shen Zhenyi Wang G. Guo Xingwei Wang Xiaocun Cao Jie Zhang Dacheng Tao MoMe 37 4 0 18 Oct 2024