Mechanistic Mode Connectivity

15 November 2022

Papers citing "Mechanistic Mode Connectivity"

50 / 110 papers shown

Title
Understanding Mode Connectivity via Parameter Space Symmetry B. Zhao Nima Dehmamy Robin Walters Rose Yu 211 7 0 29 May 2025
Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion Binchi Zhang Zaiyi Zheng Zhengzhang Chen Wenlin Yao 176 0 0 01 Feb 2025
ICLR: In-Context Learning of Representations Core Francisco Park Andrew Lee Ekdeep Singh Lubana Yongyi Yang Maya Okawa Kento Nishi Martin Wattenberg Hidenori Tanaka AIFin 201 6 0 29 Dec 2024
Task-Specific Skill Localization in Fine-tuned Language Models A. Panigrahi Nikunj Saunshi Haoyu Zhao Sanjeev Arora MoMe 59 74 0 13 Feb 2023
Characterizing Datapoints via Second-Split Forgetting Pratyush Maini Saurabh Garg Zachary Chase Lipton J. Zico Kolter 58 34 0 26 Oct 2022
On Feature Learning in the Presence of Spurious Correlations Pavel Izmailov Polina Kirichenko Nate Gruver A. Wilson 91 128 0 20 Oct 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 287 330 0 11 Sep 2022
Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning Damien Teney Maxime Peyrard Ehsan Abbasnejad 72 29 0 06 Jul 2022
Measuring Representational Robustness of Neural Networks Through Shared Invariances Vedant Nanda Till Speicher Camila Kolling John P. Dickerson Krishna P. Gummadi Adrian Weller 101 14 0 23 Jun 2022
Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization Jivat Neet Kaur Emre Kıcıman Amit Sharma UQCV OOD 64 26 0 15 Jun 2022
Linear Connectivity Reveals Generalization Strategies Jeevesh Juneja Rachit Bansal Kyunghyun Cho João Sedoc Naomi Saphra 285 45 0 24 May 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 236 133 0 19 May 2022
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations Polina Kirichenko Pavel Izmailov A. Wilson OOD 78 333 0 06 Apr 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 136 981 1 10 Mar 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 155 1,481 0 25 Feb 2022
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution Ananya Kumar Aditi Raghunathan Robbie Jones Tengyu Ma Percy Liang OODD 114 671 0 21 Feb 2022
Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization Elan Rosenfeld Pradeep Ravikumar Andrej Risteski OOD 68 81 0 14 Feb 2022
Diversify and Disambiguate: Learning From Underspecified Data Yoonho Lee Huaxiu Yao Chelsea Finn 254 66 0 07 Feb 2022
Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry Fabrizio Pittorino Antonio Ferraro Gabriele Perugini Christoph Feinauer Carlo Baldassi R. Zecchina 245 24 0 07 Feb 2022
When Do Flat Minima Optimizers Work? Jean Kaddour Linqing Liu Ricardo M. A. Silva Matt J. Kusner ODL 71 62 0 01 Feb 2022
Editing a classifier by rewriting its prediction rules Shibani Santurkar Dimitris Tsipras Mahalaxmi Elango David Bau Antonio Torralba Aleksander Madry KELM 230 90 0 02 Dec 2021
Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practices Puja Trivedi Ekdeep Singh Lubana Yujun Yan Yaoqing Yang Danai Koutra 55 55 0 05 Nov 2021
A Fine-Grained Analysis on Distribution Shift Olivia Wiles Sven Gowal Florian Stimberg Sylvestre-Alvise Rebuffi Ira Ktena Krishnamurthy Dvijotham A. Cemgil OOD 285 212 0 21 Oct 2021
Fast Model Editing at Scale E. Mitchell Charles Lin Antoine Bosselut Chelsea Finn Christopher D. Manning KELM 328 366 0 21 Oct 2021
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 77 229 0 12 Oct 2021
Distinguishing rule- and exemplar-based generalization in learning systems Ishita Dasgupta Erin Grant Thomas Griffiths 59 16 0 08 Oct 2021
Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective Luca Scimeca Seong Joon Oh Sanghyuk Chun Michael Poli Sangdoo Yun OOD 481 54 0 06 Oct 2021
Designing Counterfactual Generators using Deep Model Inversion Jayaraman J. Thiagarajan V. Narayanaswamy Deepta Rajan J. Liang Akshay S. Chaudhari A. Spanias DiffM 33 22 0 29 Sep 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 124 724 0 04 Sep 2021
Independent mechanism analysis, a new concept? Luigi Gresele Julius von Kügelgen Vincent Stimper Bernhard Schölkopf M. Besserve CML 48 102 0 09 Jun 2021
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style Julius von Kügelgen Yash Sharma Luigi Gresele Wieland Brendel Bernhard Schölkopf M. Besserve Francesco Locatello 97 313 0 08 Jun 2021
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances Berfin cSimcsek François Ged Arthur Jacot Francesco Spadaro Clément Hongler W. Gerstner Johanni Brea AI4CE 65 96 0 25 May 2021
Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks Hidenori Tanaka D. Kunin 68 29 0 06 May 2021
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges M. Bronstein Joan Bruna Taco S. Cohen Petar Velivcković GNN 350 1,148 0 27 Apr 2021
Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling Gregory W. Benton Wesley J. Maddox Sanae Lotfi A. Wilson UQCV 84 69 0 25 Feb 2021
Nonlinear Invariant Risk Minimization: A Causal Approach Chaochao Lu Yuhuai Wu Jośe Miguel Hernández-Lobato Bernhard Schölkopf CML OOD 75 51 0 24 Feb 2021
Towards Causal Representation Learning Bernhard Schölkopf Francesco Locatello Stefan Bauer Nan Rosemary Ke Nal Kalchbrenner Anirudh Goyal Yoshua Bengio OOD CML AI4CE 109 322 0 22 Feb 2021
Learning Neural Network Subspaces Mitchell Wortsman Maxwell Horton Carlos Guestrin Ali Farhadi Mohammad Rastegari UQCV 48 87 0 20 Feb 2021
When Are Solutions Connected in Deep Networks? Quynh N. Nguyen Pierre Bréchet Marco Mondelli 54 10 0 18 Feb 2021
How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation Ekdeep Singh Lubana Puja Trivedi Danai Koutra Robert P. Dick 59 15 0 04 Feb 2021
Shape or Texture: Understanding Discriminative Features in CNNs Md. Amirul Islam M. Kowal Patrick Esser Sen Jia Bjorn Ommer Konstantinos G. Derpanis Neil D. B. Bruce 69 75 0 27 Jan 2021
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics D. Kunin Javier Sagastuy-Breña Surya Ganguli Daniel L. K. Yamins Hidenori Tanaka 141 79 0 08 Dec 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 116 266 0 18 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 109 687 0 06 Nov 2020
Causal Autoregressive Flows Ilyes Khemakhem R. Monti R. Leech Aapo Hyvarinen CML OOD AI4CE 66 110 0 04 Nov 2020
On the Transfer of Disentangled Representations in Realistic Settings Andrea Dittadi Frederik Trauble Francesco Locatello M. Wuthrich Vaibhav Agrawal Ole Winther Stefan Bauer Bernhard Schölkopf OOD 98 82 0 27 Oct 2020
Linear Mode Connectivity in Multitask and Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Dilan Görür Razvan Pascanu H. Ghasemzadeh CLL 66 141 0 09 Oct 2020
What is being transferred in transfer learning? Behnam Neyshabur Hanie Sedghi Chiyuan Zhang 100 520 0 26 Aug 2020
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding David A. Klindt Lukas Schott Yash Sharma Ivan Ustyuzhaninov Wieland Brendel Matthias Bethge Dylan M. Paiton CML 87 132 0 21 Jul 2020
Measuring Robustness to Natural Distribution Shifts in Image Classification Rohan Taori Achal Dave Vaishaal Shankar Nicholas Carlini Benjamin Recht Ludwig Schmidt OOD 108 546 0 01 Jul 2020