v1v2 (latest)

Symmetries, flat minima, and the conserved quantities of gradient flow

31 October 2022

Papers citing "Symmetries, flat minima, and the conserved quantities of gradient flow"

26 / 26 papers shown

Title
Understanding Mode Connectivity via Parameter Space Symmetry B. Zhao Nima Dehmamy Robin Walters Rose Yu 225 8 0 29 May 2025
Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion Binchi Zhang Zaiyi Zheng Zhengzhang Chen Wenlin Yao 200 1 0 01 Feb 2025
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 318 344 0 11 Sep 2022
Fisher SAM: Information Geometry and Sharpness Aware Minimisation Minyoung Kim Da Li S. Hu Timothy M. Hospedales AAML 84 72 0 10 Jun 2022
Symmetry Teleportation for Accelerated Optimization B. Zhao Nima Dehmamy Robin Walters Rose Yu ODL 80 24 0 21 May 2022
Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry Fabrizio Pittorino Antonio Ferraro Gabriele Perugini Christoph Feinauer Carlo Baldassi R. Zecchina 258 26 0 07 Feb 2022
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 98 238 0 12 Oct 2021
Optimization and Sampling Under Continuous Symmetry: Examples and Lie Theory Jonathan Leake Nisheeth K. Vishnoi 133 5 0 02 Sep 2021
Noether: The More Things Change, the More Stay the Same Grzegorz Gluch R. Urbanke 48 18 0 12 Apr 2021
Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling Gregory W. Benton Wesley J. Maddox Sanae Lotfi A. Wilson UQCV 121 70 0 25 Feb 2021
Neural Teleportation M. Armenta Thierry Judge Nathan Painchaud Youssef Skandarani Carl Lemaire Gabriel Gibeau Sanchez Philippe Spino Pierre-Marc Jodoin 49 17 0 02 Dec 2020
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning Pan Zhou Jiashi Feng Chao Ma Caiming Xiong Guosheng Lin E. Weinan 98 234 0 12 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 199 1,359 0 03 Oct 2020
From Symmetry to Geometry: Tractable Nonconvex Problems Yuqian Zhang Qing Qu John N. Wright 75 45 0 14 Jul 2020
Relative Flatness and Generalization Henning Petzka Michael Kamp Linara Adilova C. Sminchisescu Mario Boley 85 78 0 03 Jan 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 163 630 0 11 Dec 2019
Asymmetric Valleys: Beyond Sharp and Flat Local Minima Haowei He Gao Huang Yang Yuan ODL MLT 69 150 0 02 Feb 2019
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 145 1,673 0 14 Mar 2018
Essentially No Barriers in Neural Network Energy Landscape Felix Dräxler K. Veschgini M. Salmhofer Fred Hamprecht MoMe 122 435 0 02 Mar 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 120 487 0 19 Feb 2018
Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes Lei Wu Zhanxing Zhu E. Weinan ODL 67 221 0 30 Jun 2017
L2 Regularization versus Batch and Weight Normalization Twan van Laarhoven 94 302 0 16 Jun 2017
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys Pratik Chaudhari A. Choromańska Stefano Soatto Yann LeCun Carlo Baldassi C. Borgs J. Chayes Levent Sagun R. Zecchina ODL 96 775 0 06 Nov 2016
Topology and Geometry of Half-Rectified Network Optimization C. Freeman Joan Bruna 233 235 0 04 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 442 2,946 0 15 Sep 2016
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 198 1,852 0 20 Dec 2013