v1v2 (latest)

AdaMerging: Adaptive Model Merging for Multi-Task Learning

4 October 2023

Li Shen

Papers citing "AdaMerging: Adaptive Model Merging for Multi-Task Learning"

37 / 37 papers shown

Title
DitHub: A Modular Framework for Incremental Open-Vocabulary Object Detection Chiara Cappellino Gianluca Mancusi Matteo Mosconi Angelo Porrello Simone Calderara Rita Cucchiara ObjD VLM 170 0 0 12 Mar 2025
Universal Incremental Learning: Mitigating Confusion from Inter- and Intra-task Distribution Randomness Sheng Luo Yi Zhou Tao Zhou CLL 159 0 0 10 Mar 2025
GNNMerge: Merging of GNN Models Without Accessing Training Data Vipul Garg Ishita Thakre Sayan Ranu MoMe 176 0 0 05 Mar 2025
LEWIS (LayEr WIse Sparsity) -- A Training Free Guided Model Merging Approach Hetarth Chopra Vidhi Rambhia Vikram Adve MoMe 135 0 0 05 Mar 2025
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation Fanhu Zeng Haiyang Guo Fei Zhu Li Shen Hao Tang MoMe 212 4 0 24 Feb 2025
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li J.N. Zhang MoMe FedML 320 2 0 18 Feb 2025
SuperMerge: An Approach For Gradient-Based Model Merging Haoyu Yang Zheng Zhang Saket Sathe MoMe 216 0 0 17 Feb 2025
Superpose Singular Features for Model Merging Haiquan Qiu You Wu Quanming Yao MoMe 164 0 0 15 Feb 2025
Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion Binchi Zhang Zaiyi Zheng Zhengzhang Chen Wenlin Yao 200 1 0 01 Feb 2025
Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts Wenju Sun Qingyong Li Wen Wang Yangli-ao Geng Boyang Li 189 5 0 28 Jan 2025
Multi-Task Model Merging via Adaptive Weight Disentanglement Feng Xiong Runxi Cheng Wang Chen Zhanqiu Zhang Yiwen Guo Chun Yuan Ruifeng Xu MoMe 201 8 0 10 Jan 2025
Task Singular Vectors: Reducing Task Interference in Model Merging Antonio Andrea Gargiulo Donato Crisostomi Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Emanuele Rodolà MoMe 147 16 0 26 Nov 2024
ATM: Improving Model Merging by Alternating Tuning and Merging Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Fabrizio Silvestri Emanuele Rodolà MoMe 133 5 0 05 Nov 2024
Closed-form merging of parameter-efficient modules for Federated Continual Learning Riccardo Salami Pietro Buzzega Matteo Mosconi Jacopo Bonato Luigi Sabetta Simone Calderara FedML MoMe CLL 105 4 0 23 Oct 2024
Agent Skill Acquisition for Large Language Models via CycleQD So Kuroki Taishi Nakamura Takuya Akiba Yujin Tang MoMe 136 2 0 16 Oct 2024
PLeaS -- Merging Models with Permutations and Least Squares Anshul Nasery J. Hayase Pang Wei Koh Sewoong Oh MoMe 110 4 0 02 Jul 2024
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging Deyuan Liu Zhan Qin Han Wang Zhao Yang Zecheng Wang ... Zhao Lv Zhiying Tu Dianhui Chu Bo Li Dianbo Sui 94 2 0 24 Jun 2024
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition Chengsong Huang Qian Liu Bill Yuchen Lin Tianyu Pang Chao Du Min Lin MoMe 107 214 0 25 Jul 2023
Editing Models with Task Arithmetic Gabriel Ilharco Marco Tulio Ribeiro Mitchell Wortsman Suchin Gururangan Ludwig Schmidt Hannaneh Hajishirzi Ali Farhadi KELM MoMe MU 203 521 0 08 Dec 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 318 344 0 11 Sep 2022
Efficient Test-Time Model Adaptation without Forgetting Shuaicheng Niu Jiaxiang Wu Yifan Zhang Yaofo Chen S. Zheng P. Zhao Mingkui Tan OOD VLM TTA 93 350 0 06 Apr 2022
MetaBalance: Improving Multi-Task Recommendations via Adapting Gradient Magnitudes of Auxiliary Tasks Yun He Xuening Feng Cheng Cheng Geng Ji Yunsong Guo James Caverlee 55 43 0 14 Mar 2022
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 504 10,526 0 17 Jun 2021
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning Hussein Hazimeh Zhe Zhao Aakanksha Chowdhery M. Sathiamoorthy Yihua Chen Rahul Mazumder Lichan Hong Ed H. Chi MoE 174 145 0 07 Jun 2021
SWAD: Domain Generalization by Seeking Flat Minima Junbum Cha Sanghyuk Chun Kyungjae Lee Han-Cheol Cho Seunghyun Park Yunsung Lee Sungrae Park MoMe 301 459 0 17 Feb 2021
Learning to Branch for Multi-Task Learning Pengsheng Guo Chen-Yu Lee Daniel Ulbricht 80 179 0 02 Jun 2020
Versatile Black-Box Optimization Jialin Liu A. Moreau Mike Preuss Baptiste Roziere Jérémy Rapin F. Teytaud O. Teytaud 49 38 0 29 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 377 1,493 0 18 Mar 2020
Gradient Surgery for Multi-Task Learning Tianhe Yu Saurabh Kumar Abhishek Gupta Sergey Levine Karol Hausman Chelsea Finn 195 1,230 0 19 Jan 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 79 60 0 07 Jan 2020
Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS Petro B. Liashchynskyi Pavlo Liashchynskyi 81 545 0 12 Dec 2019
On the Convergence of FedAvg on Non-IID Data Xiang Li Kaixuan Huang Wenhao Yang Shusen Wang Zhihua Zhang FedML 174 2,356 0 04 Jul 2019
Which Tasks Should Be Learned Together in Multi-task Learning? Trevor Scott Standley Amir Zamir Dawn Chen Leonidas Guibas Jitendra Malik Silvio Savarese 118 517 0 18 May 2019
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification P. Helber B. Bischke Andreas Dengel Damian Borth 158 1,834 0 31 Aug 2017
Remote Sensing Image Scene Classification: Benchmark and State of the Art Gong Cheng Junwei Han Xiaoqiang Lu 108 2,269 0 01 Mar 2017
How transferable are features in deep neural networks? J. Yosinski Jeff Clune Yoshua Bengio Hod Lipson OOD 238 8,363 0 06 Nov 2014
Describing Textures in the Wild Mircea Cimpoi Subhransu Maji Iasonas Kokkinos S. Mohamed Andrea Vedaldi 3DV 153 2,695 0 14 Nov 2013