Harmony in Diversity: Merging Neural Networks with Canonical Correlation
Analysis

Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis

7 July 2024

Albert Manuel Orozco Camacho

Eugene Belilovsky

Papers citing "Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis"

12 / 12 papers shown

Title
MASS: MoErging through Adaptive Subspace Selection Donato Crisostomi Alessandro Zirilli Antonio Andrea Gargiulo Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Iacopo Masi Emanuele Rodolà MoMe 40 0 0 06 Apr 2025
Model Assembly Learning with Heterogeneous Layer Weight Merging Yi-Kai Zhang Jin Wang Xu-Xiang Zhong De-Chuan Zhan Han-Jia Ye MoMe 51 0 0 27 Mar 2025
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li J.N. Zhang MoMe FedML 90 0 0 18 Feb 2025
Non-Uniform Parameter-Wise Model Merging Albert Manuel Orozco Camacho Stefan Horoi Guy Wolf Eugene Belilovsky MoMe FedML 90 0 0 20 Dec 2024
DoubleCCA: Improving Foundation Model Group Robustness with Random Sentence Embeddings Hong Liu Yitong Lu 78 0 0 25 Nov 2024
WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average Louis Fournier Adel Nabli Masih Aminbeidokhti M. Pedersoli Eugene Belilovsky Edouard Oyallon MoMe FedML 49 3 0 27 May 2024
Simultaneous linear connectivity of neural networks modulo permutation Ekansh Sharma Devin Kwok Tom Denton Daniel M. Roy David Rolnick Gintare Karolina Dziugaite 199 7 0 09 Apr 2024
PopulAtion Parameter Averaging (PAPA) Alexia Jolicoeur-Martineau Emy Gervais Kilian Fatras Yan Zhang Simon Lacoste-Julien MoMe 42 17 0 06 Apr 2023
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 314 0 11 Sep 2022
Optimizing Mode Connectivity via Neuron Alignment N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 223 80 0 05 Sep 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 296 39,198 0 01 Sep 2014