Understanding Mode Connectivity via Parameter Space Symmetry

29 May 2025

Papers citing "Understanding Mode Connectivity via Parameter Space Symmetry"

48 / 48 papers shown

Title
Paths and Ambient Spaces in Neural Loss Landscapes Daniel Dold Julius Kobialka Nicolai Palm Emanuel Sommer David Rügamer Oliver Durr AI4CE 115 0 0 05 Mar 2025
Input Space Mode Connectivity in Deep Neural Networks Jakub Vrabel Ori Shem-Ur Yaron Oz David Krueger 99 1 0 09 Sep 2024
Landscaping Linear Mode Connectivity Sidak Pal Singh Linara Adilova Michael Kamp Asja Fischer Bernhard Scholkopf Thomas Hofmann 107 6 0 24 Jun 2024
The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof Derek Lim Moe Putterman Robin Walters Haggai Maron Stefanie Jegelka 90 9 0 30 May 2024
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm Zhanpeng Zhou Zijun Chen Yilan Chen Bo Zhang Junchi Yan MoMe 78 11 0 06 Feb 2024
Disentangling Linear Mode-Connectivity Gul Sena Altintas Gregor Bachmann Lorenzo Noci Thomas Hofmann 112 8 0 15 Dec 2023
Proving Linear Mode Connectivity of Neural Networks via Optimal Transport Damien Ferbach Baptiste Goujaud Gauthier Gidel Aymeric Dieuleveut MoMe 106 16 0 29 Oct 2023
Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity Zhanpeng Zhou Yongyi Yang Xiaojiang Yang Junchi Yan Wei Hu 79 34 0 17 Jul 2023
Layer-wise Linear Mode Connectivity Linara Adilova Maksym Andriushchenko Michael Kamp Asja Fischer Martin Jaggi FedML FAtt MoMe 94 17 0 13 Jul 2023
Improving Convergence and Generalization Using Parameter Symmetries Bo Zhao Robert Mansel Gower Robin Walters Rose Yu MoMe 103 16 0 22 May 2023
Complex Critical Points of Deep Linear Neural Networks Ayush Bharadwaj Serkan Hosten 44 2 0 30 Jan 2023
Mechanistic Mode Connectivity Ekdeep Singh Lubana Eric J. Bigelow Robert P. Dick David M. Krueger Hidenori Tanaka 101 49 0 15 Nov 2022
REPAIR: REnormalizing Permuted Activations for Interpolation Repair Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 76 101 0 15 Nov 2022
Symmetries, flat minima, and the conserved quantities of gradient flow Bo Zhao I. Ganev Robin Walters Rose Yu Nima Dehmamy 77 20 0 31 Oct 2022
Random initialisations performing above chance and how to find them Frederik Benzing Simon Schug Robert Meier J. Oswald Yassir Akram Nicolas Zucchet Laurence Aitchison Angelika Steger ODL 88 26 0 15 Sep 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 293 343 0 11 Sep 2022
Linear Connectivity Reveals Generalization Strategies Jeevesh Juneja Rachit Bansal Kyunghyun Cho João Sedoc Naomi Saphra 285 47 0 24 May 2022
Symmetry Teleportation for Accelerated Optimization B. Zhao Nima Dehmamy Robin Walters Rose Yu ODL 65 24 0 21 May 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 149 1,009 1 10 Mar 2022
Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry Fabrizio Pittorino Antonio Ferraro Gabriele Perugini Christoph Feinauer Carlo Baldassi R. Zecchina 247 26 0 07 Feb 2022
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 94 237 0 12 Oct 2021
Geometry of Linear Convolutional Networks Kathlén Kohn Thomas Merkh Guido Montúfar Matthew Trager 71 20 0 03 Aug 2021
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances Berfin cSimcsek François Ged Arthur Jacot Francesco Spadaro Clément Hongler W. Gerstner Johanni Brea AI4CE 73 102 0 25 May 2021
Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling Gregory W. Benton Wesley J. Maddox Sanae Lotfi A. Wilson UQCV 92 70 0 25 Feb 2021
When Are Solutions Connected in Deep Networks? Quynh N. Nguyen Pierre Bréchet Marco Mondelli 66 10 0 18 Feb 2021
A Note on Connectivity of Sublevel Sets in Deep Learning Quynh N. Nguyen MLT 58 8 0 21 Jan 2021
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics D. Kunin Javier Sagastuy-Breña Surya Ganguli Daniel L. K. Yamins Hidenori Tanaka 144 80 0 08 Dec 2020
Optimizing Mode Connectivity via Neuron Alignment N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 278 82 0 05 Sep 2020
What is being transferred in transfer learning? Behnam Neyshabur Hanie Sedghi Chiyuan Zhang 114 527 0 26 Aug 2020
Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness Pu Zhao Pin-Yu Chen Payel Das Karthikeyan N. Ramamurthy Xue Lin AAML 123 190 0 30 Apr 2020
Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks Aleksandr Shevchenko Marco Mondelli 169 37 0 20 Dec 2019
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 159 628 0 11 Dec 2019
Model Fusion via Optimal Transport Sidak Pal Singh Martin Jaggi MoMe FedML 114 240 0 12 Oct 2019
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape Johanni Brea Berfin Simsek Bernd Illing W. Gerstner 96 58 0 05 Jul 2019
Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets Rohith Kuditipudi Xiang Wang Holden Lee Yi Zhang Zhiyuan Li Wei Hu Sanjeev Arora Rong Ge FAtt 101 93 0 14 Jun 2019
On Connected Sublevel Sets in Deep Learning Quynh N. Nguyen 104 102 0 22 Jan 2019
The loss surface of deep linear networks viewed through the algebraic geometry lens D. Mehta Tianran Chen Tingting Tang J. Hauenstein ODL 68 32 0 17 Oct 2018
Exposition and Interpretation of the Topology of Neural Networks Rickard Brüel-Gabrielsson Gunnar Carlsson FAtt 41 54 0 08 Oct 2018
Using Mode Connectivity for Loss Landscape Analysis Akhilesh Deepak Gotmare N. Keskar Caiming Xiong R. Socher 49 28 0 18 Jun 2018
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 135 1,670 0 14 Mar 2018
Essentially No Barriers in Neural Network Energy Landscape Felix Dräxler K. Veschgini M. Salmhofer Fred Hamprecht MoMe 116 434 0 02 Mar 2018
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs T. Garipov Pavel Izmailov Dmitrii Podoprikhin Dmitry Vetrov A. Wilson UQCV 89 757 0 27 Feb 2018
An introduction to Topological Data Analysis: fundamental and practical aspects for data scientists Frédéric Chazal Bertrand Michel 96 596 0 11 Oct 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 730 132,363 0 12 Jun 2017
Topology and Geometry of Half-Rectified Network Optimization C. Freeman Joan Bruna 218 235 0 04 Nov 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 776 36,881 0 25 Aug 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,426 0 10 Dec 2015
Symmetry-invariant optimization in deep networks Vijay Badrinarayanan Bamdev Mishra R. Cipolla ODL 48 34 0 05 Nov 2015