Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?

5 June 2021

Aaron Courville

Papers citing "Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?"

50 / 64 papers shown

Title
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 120 7 0 07 Nov 2023
Diverse Target and Contribution Scheduling for Domain Generalization Shaocong Long Qianyu Zhou Soham Dan Lizhuang Ma Yuan Luo 132 8 0 28 Sep 2023
Counterfactual Generative Networks Axel Sauer Andreas Geiger OOD BDL CML 90 126 0 15 Jan 2021
Does Invariant Risk Minimization Capture Invariance? Pritish Kamath Akilesh Tangella Danica J. Sutherland Nathan Srebro OOD 261 128 0 04 Jan 2021
WILDS: A Benchmark of in-the-Wild Distribution Shifts Pang Wei Koh Shiori Sagawa Henrik Marklund Sang Michael Xie Marvin Zhang ... A. Kundaje Emma Pierson Sergey Levine Chelsea Finn Percy Liang OOD 183 1,434 0 14 Dec 2020
In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness Sang Michael Xie Ananya Kumar Robbie Jones Fereshte Khani Tengyu Ma Percy Liang OOD 198 62 0 08 Dec 2020
Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately Fereshte Khani Percy Liang FaML 59 66 0 07 Dec 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 128 266 0 18 Nov 2020
Empirical or Invariant Risk Minimization? A Sample Complexity Perspective Kartik Ahuja Jun Wang Amit Dhurandhar Karthikeyan Shanmugam Kush R. Varshney OOD 83 79 0 30 Oct 2020
Understanding the Failure Modes of Out-of-Distribution Generalization Vaishnavh Nagarajan Anders Andreassen Behnam Neyshabur OOD OODD 59 177 0 29 Oct 2020
The Risks of Invariant Risk Minimization Elan Rosenfeld Pradeep Ravikumar Andrej Risteski OOD 80 312 0 12 Oct 2020
Characterising Bias in Compressed Models Sara Hooker Nyalleng Moorosi Gregory Clark Samy Bengio Emily L. Denton 67 185 0 06 Oct 2020
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks Róbert Csordás Sjoerd van Steenkiste Jürgen Schmidhuber 91 95 0 05 Oct 2020
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot Jingtong Su Yihang Chen Tianle Cai Tianhao Wu Ruiqi Gao Liwei Wang Jason D. Lee 56 86 0 22 Sep 2020
Pruning Neural Networks at Initialization: Why are We Missing the Mark? Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin 65 240 0 18 Sep 2020
Learning explanations that are hard to vary Giambattista Parascandolo Alexander Neitz Antonio Orvieto Luigi Gresele Bernhard Schölkopf FAtt 63 185 0 01 Sep 2020
Informative Dropout for Robust Representation Learning: A Shape-bias Perspective Baifeng Shi Dinghuai Zhang Qi Dai Zhanxing Zhu Yadong Mu Jingdong Wang OOD 58 112 0 10 Aug 2020
Learning from Failure: Training Debiased Classifier from Biased Classifier J. Nam Hyuntak Cha SungSoo Ahn Jaeho Lee Jinwoo Shin 63 149 0 06 Jul 2020
In Search of Lost Domain Generalization Ishaan Gulrajani David Lopez-Paz OOD 79 1,149 0 02 Jul 2020
The Pitfalls of Simplicity Bias in Neural Networks Harshay Shah Kaustav Tamuly Aditi Raghunathan Prateek Jain Praneeth Netrapalli AAML 67 361 0 13 Jun 2020
Risk Variance Penalization Chuanlong Xie Haotian Ye Fei Chen Yue Liu Rui Sun Zhenguo Li 152 33 0 13 Jun 2020
Enforcing Predictive Invariance across Structured Biomedical Domains Wengong Jin Regina Barzilay Tommi Jaakkola OOD 59 30 0 06 Jun 2020
An Analysis of the Adaptation Speed of Causal Models Rémi Le Priol Reza Babanezhad Harikandeh Yoshua Bengio Simon Lacoste-Julien CML 41 14 0 18 May 2020
An Investigation of Why Overparameterization Exacerbates Spurious Correlations Shiori Sagawa Aditi Raghunathan Pang Wei Koh Percy Liang 188 381 0 09 May 2020
Shortcut Learning in Deep Neural Networks Robert Geirhos J. Jacobsen Claudio Michaelis R. Zemel Wieland Brendel Matthias Bethge Felix Wichmann 209 2,052 0 16 Apr 2020
Invariant Rationalization Shiyu Chang Yang Zhang Mo Yu Tommi Jaakkola 235 206 0 22 Mar 2020
Out-of-Distribution Generalization via Risk Extrapolation (REx) David M. Krueger Ethan Caballero J. Jacobsen Amy Zhang Jonathan Binas Dinghuai Zhang Rémi Le Priol Aaron Courville OOD 307 936 0 02 Mar 2020
Invariant Risk Minimization Games Kartik Ahuja Karthikeyan Shanmugam Kush R. Varshney Amit Dhurandhar OOD 65 250 0 11 Feb 2020
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization Shiori Sagawa Pang Wei Koh Tatsunori B. Hashimoto Percy Liang OOD 108 1,243 0 20 Nov 2019
What Do Compressed Deep Neural Networks Forget? Sara Hooker Aaron Courville Gregory Clark Yann N. Dauphin Andrea Frome 87 185 0 13 Nov 2019
Learning De-biased Representations with Biased Representations Hyojin Bahng Sanghyuk Chun Sangdoo Yun Jaegul Choo Seong Joon Oh OOD 381 281 0 07 Oct 2019
Recurrent Independent Mechanisms Anirudh Goyal Alex Lamb Jordan Hoffmann Shagun Sodhani Sergey Levine Yoshua Bengio Bernhard Schölkopf 89 337 0 24 Sep 2019
Invariant Risk Minimization Martín Arjovsky Léon Bottou Ishaan Gulrajani David Lopez-Paz OOD 192 2,229 0 05 Jul 2019
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask Hattie Zhou Janice Lan Rosanne Liu J. Yosinski UQCV 55 387 0 03 May 2019
Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet Wieland Brendel Matthias Bethge SSL FAtt 94 561 0 20 Mar 2019
Learning Robust Representations by Projecting Superficial Statistics Out Haohan Wang Zexue He Zachary Chase Lipton Eric Xing OOD 70 235 0 02 Mar 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 76 314 0 15 Feb 2019
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms Yoshua Bengio T. Deleu Nasim Rahaman Nan Rosemary Ke Sébastien Lachapelle O. Bilaniuk Anirudh Goyal C. Pal CML OOD 107 334 0 30 Jan 2019
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness Robert Geirhos Patricia Rubisch Claudio Michaelis Matthias Bethge Felix Wichmann Wieland Brendel 103 2,670 0 29 Nov 2018
Rethinking the Value of Network Pruning Zhuang Liu Mingjie Sun Tinghui Zhou Gao Huang Trevor Darrell 36 1,470 0 11 Oct 2018
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 263 1,205 0 04 Oct 2018
Recognition in Terra Incognita Sara Beery Grant Van Horn Pietro Perona 92 851 0 13 Jul 2018
Automatically Composing Representation Transformations as a Means for Generalization Michael Chang Abhishek Gupta Sergey Levine Thomas Griffiths 47 69 0 12 Jul 2018
Robustness May Be at Odds with Accuracy Dimitris Tsipras Shibani Santurkar Logan Engstrom Alexander Turner Aleksander Madry AAML 104 1,781 0 30 May 2018
Understanding Community Structure in Layered Neural Networks C. Watanabe Kaoru Hiramatsu K. Kashino 123 22 0 13 Apr 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 235 3,473 0 09 Mar 2018
Learning Sparse Neural Networks through $L_0$ Regularization Christos Louizos Max Welling Diederik P. Kingma 433 1,144 0 04 Dec 2017
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning Arun Mallya Svetlana Lazebnik CLL 105 1,301 0 15 Nov 2017
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 158 917 0 27 Oct 2017
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 194 1,278 0 05 Oct 2017