Deep Networks on Toroids: Removing Symmetries Reveals the Structure of
Flat Regions in the Landscape Geometry

Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry

7 February 2022

Fabrizio Pittorino

Antonio Ferraro

Gabriele Perugini

Christoph Feinauer

Papers citing "Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry"

10 / 10 papers shown

Title
The Empirical Impact of Reducing Symmetries on the Performance of Deep Ensembles and MoE Andrei Chernov Oleg Novitskij 48 0 0 24 Feb 2025
High-dimensional manifold of solutions in neural networks: insights from statistical physics Enrico M. Malatesta 51 4 0 20 Feb 2025
FlatNAS: optimizing Flatness in Neural Architecture Search for Out-of-Distribution Robustness Matteo Gambella Fabrizio Pittorino Manuel Roveri OOD 42 3 0 29 Feb 2024
Mechanistic Mode Connectivity Ekdeep Singh Lubana Eric J. Bigelow Robert P. Dick David M. Krueger Hidenori Tanaka 32 45 0 15 Nov 2022
REPAIR: REnormalizing Permuted Activations for Interpolation Repair Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 46 94 0 15 Nov 2022
Symmetries, flat minima, and the conserved quantities of gradient flow Bo Zhao I. Ganev Robin Walters Rose Yu Nima Dehmamy 47 16 0 31 Oct 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 316 0 11 Sep 2022
Learning through atypical "phase transitions" in overparameterized neural networks Carlo Baldassi Clarissa Lauditi Enrico M. Malatesta R. Pacelli Gabriele Perugini R. Zecchina 26 26 0 01 Oct 2021
Optimizing Mode Connectivity via Neuron Alignment N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 223 80 0 05 Sep 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016