Understanding Learning Invariance in Deep Linear Networks

16 June 2025

Papers citing "Understanding Learning Invariance in Deep Linear Networks"

39 / 39 papers shown

Title
Emergent Equivariance in Deep Ensembles Jan E. Gerken Pan Kessel UQCV MDE 64 8 0 05 Mar 2024
Algebraic Complexity and Neurovariety of Linear Convolutional Networks Vahid Shahverdi 103 4 0 29 Jan 2024
Should Under-parameterized Student Networks Copy or Average Teacher Weights? Berfin Simsek Amire Bendjeddou W. Gerstner Johanni Brea 61 8 0 03 Nov 2023
Geometry of Linear Neural Networks: Equivariance and Invariance under Permutation Groups Kathlén Kohn Anna-Laura Sattelberger Vahid Shahverdi 65 4 0 24 Sep 2023
On genuine invariance learning without weight-tying A. Moskalev A. Sepliarskaia Erik J. Bekkers A. Smeulders CML OOD 48 9 0 07 Aug 2023
EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations Yidong Liao Brandon M. Wood Abhishek Das Tess E. Smidt 96 160 0 21 Jun 2023
Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape Kedar Karhadkar Michael Murray Hanna Tseran Guido Montúfar 49 8 0 31 May 2023
Function Space and Critical Points of Linear Convolutional Networks Kathlén Kohn Guido Montúfar Vahid Shahverdi Matthew Trager 57 13 0 12 Apr 2023
The Exact Sample Complexity Gain from Invariances for Kernel Regression B. Tahmasebi Stefanie Jegelka 56 19 0 24 Mar 2023
Optimization Dynamics of Equivariant and Augmented Neural Networks Axel Flinth F. Ohlsson 84 7 0 23 Mar 2023
Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss Pierre Bréchet Katerina Papagiannouli Jing An Guido Montúfar 55 4 0 06 Mar 2023
Equivariant Polynomials for Graph Neural Networks Omri Puny Derek Lim B. Kiani Haggai Maron Y. Lipman 71 33 0 22 Feb 2023
Symmetries, flat minima, and the conserved quantities of gradient flow Bo Zhao I. Ganev Robin Walters Rose Yu Nima Dehmamy 77 20 0 31 Oct 2022
How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularization Jonas Geiping Micah Goldblum Gowthami Somepalli Ravid Shwartz-Ziv Tom Goldstein A. Wilson 72 43 0 12 Oct 2022
Overparameterization from Computational Constraints Sanjam Garg S. Jha Saeed Mahloujifar Mohammad Mahmoody Mingyuan Wang 47 2 0 27 Aug 2022
Geometry of Linear Convolutional Networks Kathlén Kohn Thomas Merkh Guido Montúfar Matthew Trager 64 20 0 03 Aug 2021
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances Berfin cSimcsek François Ged Arthur Jacot Francesco Spadaro Clément Hongler W. Gerstner Johanni Brea AI4CE 73 102 0 25 May 2021
A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups Marc Finzi Max Welling A. Wilson 162 197 0 19 Apr 2021
Learning with invariances in random features and kernel models Song Mei Theodor Misiakiewicz Andrea Montanari OOD 100 91 0 25 Feb 2021
An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage C. L. Zitnick L. Chanussot Abhishek Das Siddharth Goyal Javier Heras-Domingo ... Kevin Tran Brandon M. Wood Junwoong Yoon Devi Parikh Zachary W. Ulissi 63 75 0 14 Oct 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 532 42,559 0 03 Dec 2019
Enhanced Convolutional Neural Tangent Kernels Zhiyuan Li Ruosong Wang Dingli Yu S. Du Wei Hu Ruslan Salakhutdinov Sanjeev Arora 68 133 0 03 Nov 2019
Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers B. Bah Holger Rauhut Ulrich Terstiege Michael Westdickenberg MLT 37 66 0 12 Oct 2019
Pure and Spurious Critical Points: a Geometric Study of Linear Networks Matthew Trager Kathlén Kohn Joan Bruna 52 31 0 03 Oct 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 87 509 0 31 May 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 242 1,655 0 28 Dec 2018
Invariant and Equivariant Graph Networks Haggai Maron Heli Ben-Hamu Nadav Shamir Y. Lipman 141 507 0 24 Dec 2018
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks Sanjeev Arora Nadav Cohen Noah Golowich Wei Hu 125 293 0 04 Oct 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 273 3,219 0 20 Jun 2018
Implicit Bias of Gradient Descent on Linear Convolutional Networks Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro MDE 124 413 0 01 Jun 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 105 488 0 19 Feb 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 258 1,896 0 28 Dec 2017
Deep linear neural networks with arbitrary loss: All local minima are global T. Laurent J. V. Brecht ODL 71 137 0 05 Dec 2017
Theoretical insights into the optimization landscape of over-parameterized shallow neural networks Mahdi Soltanolkotabi Adel Javanmard Jason D. Lee 175 423 0 16 Jul 2017
Low-Rank Matrix Approximation in the Infinity Norm Nicolas Gillis Y. Shitov 49 27 0 31 May 2017
Deep Sets Manzil Zaheer Satwik Kottur Siamak Ravanbakhsh Barnabás Póczós Ruslan Salakhutdinov Alex Smola 421 2,478 0 10 Mar 2017
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 224 925 0 23 May 2016
Group Equivariant Convolutional Networks Taco S. Cohen Max Welling BDL 171 1,945 0 24 Feb 2016
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 183 1,852 0 20 Dec 2013