v1v2v3 (latest)

Training-Free Pretrained Model Merging

4 March 2024

ArXiv (abs)PDF HTML Github (28★)

Papers citing "Training-Free Pretrained Model Merging"

23 / 23 papers shown

Title
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li J.N. Zhang MoMe FedML 308 2 0 18 Feb 2025
Training-free Heterogeneous Model Merging Zhengqi Xu Han Zheng Jie Song Li Sun Mingli Song MoMe 232 1 0 03 Jan 2025
Extracting and Transferring Abilities For Building Multi-lingual Ability-enhanced Large Language Models Zhipeng Chen Liang Song K. Zhou Wayne Xin Zhao Binghai Wang Weipeng Chen Ji-Rong Wen 117 0 0 10 Oct 2024
PLeaS -- Merging Models with Permutations and Least Squares Anshul Nasery J. Hayase Pang Wei Koh Sewoong Oh MoMe 95 3 0 02 Jul 2024
Contrastive Knowledge Amalgamation for Unsupervised Image Classification Shangde Gao Yichao Fu Li-Yu Daisy Liu Yuqiang Han 42 8 0 27 Jul 2023
Hidden symmetries of ReLU networks J. E. Grigsby Kathryn A. Lindsey David Rolnick 53 23 0 09 Jun 2023
Re-basin via implicit Sinkhorn differentiation F. Guerrero-Peña H. R. Medeiros Thomas Dubail Masih Aminbeidokhti Eric Granger M. Pedersoli MoMe 73 49 0 22 Dec 2022
Editing Models with Task Arithmetic Gabriel Ilharco Marco Tulio Ribeiro Mitchell Wortsman Suchin Gururangan Ludwig Schmidt Hannaneh Hajishirzi Ali Farhadi KELM MoMe MU 185 496 0 08 Dec 2022
REPAIR: REnormalizing Permuted Activations for Interpolation Repair Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 71 99 0 15 Nov 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 290 335 0 11 Sep 2022
Similarity and Matching of Neural Network Representations Adrián Csiszárik Péter Korösi-Szabó Á. Matszangosz Gergely Papp D. Varga 44 70 0 27 Oct 2021
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 91 236 0 12 Oct 2021
Efficiently Identifying Task Groupings for Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 284 254 1 10 Sep 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 694 6,079 0 29 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 453 21,439 0 25 Mar 2021
Optimizing Mode Connectivity via Neuron Alignment N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 272 82 0 05 Sep 2020
Model Fusion via Optimal Transport Sidak Pal Singh Martin Jaggi MoMe FedML 109 237 0 12 Oct 2019
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape Johanni Brea Berfin Simsek Bernd Illing W. Gerstner 94 57 0 05 Jul 2019
Taskonomy: Disentangling Task Transfer Learning Amir Zamir Alexander Sax Bokui (William) Shen Leonidas Guibas Jitendra Malik Silvio Savarese 123 1,220 0 23 Apr 2018
Group Normalization Yuxin Wu Kaiming He 231 3,660 0 22 Mar 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 240 3,473 0 09 Mar 2018
A Survey on Multi-Task Learning Yu Zhang Qiang Yang AIMat 605 2,230 0 25 Jul 2017
Understanding image representations by measuring their equivariance and equivalence Karel Lenc Andrea Vedaldi SSL FAtt 112 533 0 21 Nov 2014