Merging Models with Fisher-Weighted Averaging

18 November 2021

Papers citing "Merging Models with Fisher-Weighted Averaging"

50 / 283 papers shown

Title
Mergenetic: a Simple Evolutionary Model Merging Library Adrian Robert Minut Tommaso Mencattini Andrea Santilli Donato Crisostomi Emanuele Rodolà MoMe 19 0 0 16 May 2025
MergeBench: A Benchmark for Merging Domain-Specialized LLMs Yifei He Siqi Zeng Yuzheng Hu Rui Yang Tong Zhang Han Zhao MoMe ALM 21 0 0 16 May 2025
CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging Wenju Sun Qingyong Li Yangli-ao Geng Boyang Li MoMe 34 0 0 11 May 2025
QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration HamidReza Imani Jiaxin Peng Peiman Mohseni Abdolah Amirany Tarek A. El-Ghazawi MoE 31 0 0 10 May 2025
Dynamic Fisher-weighted Model Merging via Bayesian Optimization Sanwoo Lee Jiahao Liu Qifan Wang J. Wang Xunliang Cai Yunfang Wu MoMe 148 0 0 26 Apr 2025
PatientDx: Merging Large Language Models for Protecting Data-Privacy in Healthcare José G. Moreno Jesus Lovon M'Rick Robin-Charlet Christine Damase-Michel L. Tamine MoMe LM&MA 58 0 0 24 Apr 2025
Param $Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost Sheng Cao Mingrui Wu Karthik Prasad Yuandong Tian Zechun Liu MoMe 80 0 0 23 Apr 2025
Mitigating Parameter Interference in Model Merging via Sharpness-Aware Fine-Tuning Yeoreum Lee Jinwook Jung Sungyong Baik MoMe 40 0 0 20 Apr 2025
ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs Yan Yang Yixia Li Hongru Wang Xuetao Wei Jianqiao Yu Yun-Nung Chen Guanhua Chen MoMe 28 0 0 17 Apr 2025
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers Hongkang Li Yihua Zhang Shuai Zhang Hao Wu Sijia Liu Pin-Yu Chen MoMe 69 3 0 15 Apr 2025
Leveraging Submodule Linearity Enhances Task Arithmetic Performance in LLMs Rui Dai Sile Hu Xu Shen Yonggang Zhang Xinmei Tian Jieping Ye MoMe 49 2 0 15 Apr 2025
Metropolis-Hastings Captioning Game: Knowledge Fusion of Vision Language Models via Decentralized Bayesian Inference Yuta Matsui Ryosuke Yamaki Ryo Ueda Seitaro Shinagawa Tadahiro Taniguchi MLLM 38 1 0 13 Apr 2025
LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation Juzheng Zhang Jiacheng You Ashwinee Panda Tom Goldstein MoMe 53 0 0 10 Apr 2025
FedMerge: Federated Personalization via Model Merging Shutong Chen Tianyi Zhou Guodong Long Jing Jiang Chengqi Zhang FedML MoMe 49 0 0 09 Apr 2025
Defending Deep Neural Networks against Backdoor Attacks via Module Switching Weijun Li Ansh Arora Xuanli He Mark Dras Qiongkai Xu AAML MoMe 53 0 0 08 Apr 2025
Exact Unlearning of Finetuning Data via Model Merging at Scale Kevin Kuo Amrith Rajagopal Setlur Kartik Srinivas Aditi Raghunathan Virginia Smith MoMe CLL MU 45 0 0 06 Apr 2025
MASS: MoErging through Adaptive Subspace Selection Donato Crisostomi Alessandro Zirilli Antonio Andrea Gargiulo Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Iacopo Masi Emanuele Rodolà MoMe 40 0 0 06 Apr 2025
Efficient Model Editing with Task-Localized Sparse Fine-tuning Leonardo Iurada Marco Ciccone Tatiana Tommasi KELM MoMe 56 1 0 03 Apr 2025
BECAME: BayEsian Continual Learning with Adaptive Model MErging Mei Li Yuxiang Lu Qinyan Dai Suizhi Huang Yue Ding Hongtao Lu CLL MoMe 49 0 0 03 Apr 2025
SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearning Tianyang Xu Xiaoze Liu Feijie Wu Xiaoqian Wang Jing Gao MU 58 0 0 29 Mar 2025
AdaRank: Adaptive Rank Pruning for Enhanced Model Merging Chanhyuk Lee Jiho Choi Chanryeol Lee Donggyun Kim Seunghoon Hong MoMe 52 0 0 28 Mar 2025
Fusion of Graph Neural Networks via Optimal Transport Weronika Ormaniec Michael Vollenweider Elisa Hoskovec MoMe FedML OT 70 0 0 27 Mar 2025
Model Assembly Learning with Heterogeneous Layer Weight Merging Yi-Kai Zhang Jin Wang Xu-Xiang Zhong De-Chuan Zhan Han-Jia Ye MoMe 47 0 0 27 Mar 2025
FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation Dong Zhao Jinlong Li Shuang Wang Mengyao Wu Qi Zang N. Sebe Zhun Zhong 147 0 0 23 Mar 2025
SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging Aladin Djuhera S. Kadhe Farhan Ahmed Syed Zawad Holger Boche MoMe 51 0 0 21 Mar 2025
FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors Changlong Shi He Zhao Bingjie Zhang Mingyuan Zhou Dandan Guo Yi Chang 44 0 0 20 Mar 2025
FedBEns: One-Shot Federated Learning based on Bayesian Ensemble Jacopo Talpini Marco Savi Giovanni Neglia FedML Presented at ResearchTrend Connect \| FedML on 07 May 2025 76 0 0 19 Mar 2025
When Domain Generalization meets Generalized Category Discovery: An Adaptive Task-Arithmetic Driven Approach Vaibhav Rathore S. Bagchi Saikat Dutta Sarthak Mehrotra Zsolt Kira Biplab Banerjee OOD 76 1 0 19 Mar 2025
FedVSR: Towards Model-Agnostic Federated Learning in Video Super-Resolution Ali Mollaahmadi Dehaghi Hossein KhademSohi Reza Razavi Steve Drew Mohammad Moshirpour FedML 56 0 0 17 Mar 2025
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization Hao Chen S. Hu Wayne Luk Timothy M. Hospedales Hongxiang Fan MoMe 72 0 0 16 Mar 2025
From Task-Specific Models to Unified Systems: A Review of Model Merging Approaches Wei Ruan Tianze Yang Yue Zhou Tianming Liu Jin Lu MoMe 90 0 0 13 Mar 2025
Enhanced Continual Learning of Vision-Language Models with Model Fusion Haoyuan Gao Zicong Zhang Yuqi Wei Linglan Zhao Guilin Li Y. Li Linghe Kong Weiran Huang CLL VLM 167 0 0 12 Mar 2025
Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors Runxi Cheng Feng Xiong Yongxian Wei Wanyun Zhu Chun Yuan MoMe 68 0 0 11 Mar 2025
RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs Zhongzhan Huang Guoming Ling Vincent S. Liang Yupei Lin Yandong Chen Shanshan Zhong Hefeng Wu Liang Lin LRM 54 2 0 08 Mar 2025
SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting Linqi Yang Xiongwei Zhao Qihao Sun Ke Wang Ao Chen Peng Kang 3DGS 80 0 0 07 Mar 2025
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model Wenke Huang Jian Liang Xianda Guo Yiyang Fang Guancheng Wan ... Bin Yang He Li Jiawei Shao Mang Ye Bo Du OffRL LRM MLLM KELM VLM 65 1 0 06 Mar 2025
Extrapolation Merging: Keep Improving With Extrapolation and Merging Yiguan Lin Bin Xu Yinghao Li Yang Gao MoMe 59 1 0 05 Mar 2025
GNNMerge: Merging of GNN Models Without Accessing Training Data Vipul Garg Ishita Thakre Sayan Ranu MoMe 82 0 0 05 Mar 2025
LEWIS (LayEr WIse Sparsity) -- A Training Free Guided Model Merging Approach Hetarth Chopra Vidhi Rambhia Vikram Adve MoMe 70 0 0 05 Mar 2025
Med-LEGO: Editing and Adapting toward Generalist Medical Image Diagnosis Yitao Zhu Yuan Yin Jiaming Li Mengjie Xu Zihao Zhao Honglin Xiong Sheng Wang Qian Wang MedIm 72 0 0 03 Mar 2025
Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think Jie Tian Xiaoye Qu Zhenyi Lu Wei Wei Sichen Liu Yu-Xi Cheng DiffM VGen 44 0 0 02 Mar 2025
Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge Yan-Lun Chen Yi-Ru Wei Chia-Yi Hsu Chia-Mu Yu Chun-ying Huang Ying-Dar Lin Yu-Sung Wu Wei-Bin Lee MoMe KELM 56 0 0 27 Feb 2025
CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Merging Zongzhen Yang Binhang Qi Hailong Sun Wenrui Long Ruobing Zhao Xiang Gao MoMe 48 0 0 26 Feb 2025
CAMEx: Curvature-aware Merging of Experts Dung V. Nguyen Minh H. Nguyen Luc Q. Nguyen R. Teo T. Nguyen Linh Duy Tran MoMe 104 2 0 26 Feb 2025
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation Fanhu Zeng Haiyang Guo Fei Zhu Li Shen Hao Tang MoMe 54 1 0 24 Feb 2025
LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint Qianli Ma Dongrui Liu Qian Chen Linfeng Zhang Jing Shao MoMe 147 0 0 24 Feb 2025
Delta Decompression for MoE-based LLMs Compression Hao Gu Wei Li Lujun Li Qiyuan Zhu Mark Lee Shengjie Sun Wei Xue Yike Guo MoE 52 0 0 24 Feb 2025
Mixup Model Merge: Enhancing Model Merging Performance through Randomized Linear Interpolation Yue Zhou Yi-Ju Chang Yuan Wu MoMe 66 2 0 24 Feb 2025
MedForge: Building Medical Foundation Models Like Open Source Software Development Zheling Tan Kexin Ding Jin Gao Mu Zhou Dimitris N. Metaxas Shaoting Zhang Dequan Wang AI4CE 45 1 0 22 Feb 2025
Robust Concept Erasure Using Task Vectors Minh Pham Kelly O. Marshall Chinmay Hegde Niv Cohen 120 17 0 21 Feb 2025