Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry

7 February 2022

Papers citing "Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry"

9 / 9 papers shown

Title
The Empirical Impact of Reducing Symmetries on the Performance of Deep Ensembles and MoE Andrei Chernov Oleg Novitskij 48 0 0 24 Feb 2025
High-dimensional manifold of solutions in neural networks: insights from statistical physics Enrico M. Malatesta 46 4 0 20 Feb 2025
FlatNAS: optimizing Flatness in Neural Architecture Search for Out-of-Distribution Robustness Matteo Gambella Fabrizio Pittorino Manuel Roveri OOD 42 3 0 29 Feb 2024
REPAIR: REnormalizing Permuted Activations for Interpolation Repair Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 46 94 0 15 Nov 2022
Symmetries, flat minima, and the conserved quantities of gradient flow Bo-Lu Zhao I. Ganev Robin G. Walters Rose Yu Nima Dehmamy 47 16 0 31 Oct 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 252 314 0 11 Sep 2022
Learning through atypical "phase transitions" in overparameterized neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta R. Pacelli Gabriele Perugini R. Zecchina 26 26 0 01 Oct 2021
Optimizing Mode Connectivity via Neuron Alignment N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 223 80 0 05 Sep 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 284 2,889 0 15 Sep 2016