Optimizing Mode Connectivity via Neuron Alignment

5 September 2020

Papers citing "Optimizing Mode Connectivity via Neuron Alignment"

35 / 35 papers shown

Title
Understanding Mode Connectivity via Parameter Space Symmetry B. Zhao Nima Dehmamy Robin Walters Rose Yu 63 7 0 29 May 2025
Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models Shuqi Liu Han Wu Bowei He Xiongwei Han Mingxuan Yuan Linqi Song MoMe 91 3 0 20 Feb 2025
Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion Binchi Zhang Zaiyi Zheng Zhengzhang Chen Wenlin Yao 99 0 0 01 Feb 2025
Merging Feed-Forward Sublayers for Compressed Transformers Neha Verma Kenton W. Murray Kevin Duh AI4CE 79 0 0 10 Jan 2025
Training-free Heterogeneous Model Merging Zhengqi Xu Han Zheng Jie Song Li Sun Mingli Song MoMe 142 1 0 03 Jan 2025
Arcee's MergeKit: A Toolkit for Merging Large Language Models Charles Goddard Shamane Siriwardhana Malikeh Ehghaghi Luke Meyers Vladimir Karpukhin Brian Benedict Mark McQuade Jacob Solawetz MoMe KELM 99 92 0 20 Mar 2024
Deterministic Nonsmooth Nonconvex Optimization Michael I. Jordan Guy Kornowski Tianyi Lin Ohad Shamir Manolis Zampetakis 80 26 0 16 Feb 2023
Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness Pu Zhao Pin-Yu Chen Payel Das Karthikeyan N. Ramamurthy Xue Lin AAML 90 187 0 30 Apr 2020
Overfitting in adversarially robust deep learning Leslie Rice Eric Wong Zico Kolter 62 794 0 26 Feb 2020
Model Fusion via Optimal Transport Sidak Pal Singh Martin Jaggi MoMe FedML 60 231 0 12 Oct 2019
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape Johanni Brea Berfin Simsek Bernd Illing W. Gerstner 59 55 0 05 Jul 2019
Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets Rohith Kuditipudi Xiang Wang Holden Lee Yi Zhang Zhiyuan Li Wei Hu Sanjeev Arora Rong Ge FAtt 72 93 0 14 Jun 2019
Similarity of Neural Network Representations Revisited Simon Kornblith Mohammad Norouzi Honglak Lee Geoffrey E. Hinton 105 1,382 0 01 May 2019
Robustness via curvature regularization, and vice versa Seyed-Mohsen Moosavi-Dezfooli Alhussein Fawzi J. Uesato P. Frossard AAML 52 319 0 23 Nov 2018
Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation Liwei Wang Lunjia Hu Jiayuan Gu Y. Wu Zhiqiang Hu Kun He John E. Hopcroft SSL 24 113 0 28 Oct 2018
Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models D. Su Huan Zhang Hongge Chen Jinfeng Yi Pin-Yu Chen Yupeng Gao VLM 76 390 0 05 Aug 2018
Using Mode Connectivity for Loss Landscape Analysis Akhilesh Deepak Gotmare N. Keskar Caiming Xiong R. Socher 17 27 0 18 Jun 2018
Insights on representational similarity in neural networks with canonical correlation Ari S. Morcos M. Raghu Samy Bengio DRL 38 440 0 14 Jun 2018
Essentially No Barriers in Neural Network Energy Landscape Felix Dräxler K. Veschgini M. Salmhofer Fred Hamprecht MoMe 95 430 0 02 Mar 2018
Computational Optimal Transport Gabriel Peyré Marco Cuturi OT 108 2,133 0 01 Mar 2018
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs T. Garipov Pavel Izmailov Dmitrii Podoprikhin Dmitry Vetrov A. Wilson UQCV 47 746 0 27 Feb 2018
Riemannian approach to batch normalization Minhyung Cho Jaehyung Lee 43 94 0 27 Sep 2017
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 181 11,962 0 19 Jun 2017
Wasserstein GAN Martín Arjovsky Soumith Chintala Léon Bottou GAN 105 4,817 0 26 Jan 2017
Topology and Geometry of Half-Rectified Network Optimization C. Freeman Joan Bruna 110 235 0 04 Nov 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 998 192,638 0 10 Dec 2015
Proximal gradient method for huberized support vector machine Yangyang Xu I. Akrotirianakis A. Chakraborty 18 25 0 30 Nov 2015
Convergent Learning: Do different neural networks learn the same representations? Yixuan Li J. Yosinski Jeff Clune Hod Lipson John E. Hopcroft SSL 69 358 0 24 Nov 2015
On the Quality of the Initial Basin in Overspecified Neural Networks Itay Safran Ohad Shamir 39 127 0 13 Nov 2015
Explorations on high dimensional landscapes Levent Sagun V. U. Güney Gerard Ben Arous Yann LeCun 32 65 0 20 Dec 2014
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 122 18,922 0 20 Dec 2014
Qualitatively characterizing neural network optimization problems Ian Goodfellow Oriol Vinyals Andrew M. Saxe ODL 73 519 0 19 Dec 2014
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 218 1,189 0 30 Nov 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 222 43,511 0 17 Sep 2014
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization Yann N. Dauphin Razvan Pascanu Çağlar Gülçehre Kyunghyun Cho Surya Ganguli Yoshua Bengio ODL 84 1,379 0 10 Jun 2014