Gradient Starvation: A Learning Proclivity in Neural Networks

18 November 2020

Aaron Courville

Papers citing "Gradient Starvation: A Learning Proclivity in Neural Networks"

47 / 47 papers shown

Title
Predicting Practically? Domain Generalization for Predictive Analytics in Real-world Environments Hanyu Duan Yi Yang Ahmed Abbasi Kar Yan Tam OOD 95 0 0 05 Mar 2025
Do ImageNet-trained models learn shortcuts? The impact of frequency shortcuts on generalization Shunxin Wang Raymond N. J. Veldhuis N. Strisciuglio VLM 71 0 0 05 Mar 2025
A Lightweight and Extensible Cell Segmentation and Classification Model for Whole Slide Images N. Shvetsov T. Kilvaer M. Tafavvoghi Anders Sildnes Kajsa Møllersen Lill-ToveRasmussen Busund L. A. Bongo VLM 71 1 0 26 Feb 2025
FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groups Géraldin Nanfack Eugene Belilovsky 59 0 0 10 Feb 2025
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 77 3 0 05 Jun 2024
Towards a Better Evaluation of Out-of-Domain Generalization Duhun Hwang Suhyun Kang Moonjung Eo Jimyeong Kim Wonjong Rhee 56 0 0 30 May 2024
Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlations GuanWen Qiu Da Kuang Surbhi Goel 27 8 0 05 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 100 18 0 04 Mar 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 42 1 0 01 Mar 2024
Evolutionary algorithms as an alternative to backpropagation for supervised training of Biophysical Neural Networks and Neural ODEs James Hazelden Yuhan Helena Liu Eli Shlizerman E. Shea-Brown 39 2 0 17 Nov 2023
Domain Generalization in Computational Pathology: Survey and Guidelines Mostafa Jahanifar M. Raza Kesi Xu T. Vuong R. Jewsbury ... Neda Zamanitajeddin Jin Tae Kwak S. Raza F. Minhas Nasir M. Rajpoot OOD 28 17 0 30 Oct 2023
Bayesian Domain Invariant Learning via Posterior Generalization of Parameter Distributions Shiyu Shen Bin Pan Tianyang Shi Tao Li Zhenwei Shi BDL OOD 29 1 0 25 Oct 2023
Bias Amplification Enhances Minority Group Performance Gaotang Li Jiarui Liu Wei Hu 28 5 0 13 Sep 2023
Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods A. Ma Yangchen Pan Amir-massoud Farahmand AAML 25 5 0 13 Aug 2023
Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts Annie S. Chen Yoonho Lee Amrith Rajagopal Setlur Sergey Levine Chelsea Finn OOD 16 5 0 19 Jun 2023
Consistency Regularization for Domain Generalization with Logit Attribution Matching Han Gao Kaican Li Weiyan Xie Zhi Lin Yongxiang Huang Luning Wang Caleb Chen Cao N. Zhang 13 2 0 13 May 2023
Implicit Visual Bias Mitigation by Posterior Estimate Sharpening of a Bayesian Neural Network Rebecca S Stone Nishant Ravikumar A. Bulpitt David C. Hogg BDL 36 0 0 29 Mar 2023
Finding Competence Regions in Domain Generalization Jens Müller Stefan T. Radev R. Schmier Felix Dräxler Carsten Rother Ullrich Kothe 19 4 0 17 Mar 2023
Delving into Identify-Emphasize Paradigm for Combating Unknown Bias Bowen Zhao Chen Chen Qian-Wei Wang Anfeng He Shutao Xia 29 1 0 22 Feb 2023
Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Features Annie S. Chen Yoonho Lee Amrith Rajagopal Setlur Sergey Levine Chelsea Finn VLM 29 9 0 10 Feb 2023
Look Beyond Bias with Entropic Adversarial Data Augmentation Thomas Duboudin Emmanuel Dellandréa Corentin Abgrall Gilles Hénaff Liming Luke Chen CML 35 4 0 10 Jan 2023
Learning useful representations for shifting tasks and distributions Jianyu Zhang Léon Bottou OOD 34 13 0 14 Dec 2022
A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others Zhiheng Li Ivan Evtimov Albert Gordo C. Hazirbas Tal Hassner Cristian Canton Ferrer Chenliang Xu Mark Ibrahim 34 71 0 09 Dec 2022
Outlier-Aware Training for Improving Group Accuracy Disparities Li-Kuang Chen Canasai Kruengkrai Junichi Yamagishi 24 0 0 27 Oct 2022
On Feature Learning in the Presence of Spurious Correlations Pavel Izmailov Polina Kirichenko Nate Gruver A. Wilson 34 117 0 20 Oct 2022
Learning Less Generalizable Patterns with an Asymmetrically Trained Double Classifier for Better Test-Time Adaptation Thomas Duboudin Emmanuel Dellandréa Corentin Abgrall Gilles Hénaff Limin Chen TTA 27 1 0 17 Oct 2022
MaskTune: Mitigating Spurious Correlations by Forcing to Explore Saeid Asgari Taghanaki Aliasghar Khani Fereshte Khani A. Gholami Linh-Tam Tran Ali Mahdavi-Amiri Ghassan Hamarneh AAML 41 45 0 30 Sep 2022
Artifact-Based Domain Generalization of Skin Lesion Models Alceu Bissoto Catarina Barata Eduardo Valle Sandra Avila MedIm AI4CE 38 13 0 20 Aug 2022
Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning Damien Teney Maxime Peyrard Ehsan Abbasnejad 35 29 0 06 Jul 2022
CDNet: Contrastive Disentangled Network for Fine-Grained Image Categorization of Ocular B-Scan Ultrasound Ruilong Dan Yunxiang Li Yijie Wang Gangyong Jia Ruiquan Ge Juan Ye Qun Jin Yaqi Wang 23 8 0 17 Jun 2022
Evolving Domain Generalization Wei Wang Gezheng Xu Ruizhi Pu Jiaqi Li Fan Zhou Changjian Shui Charles Ling Christian Gagné Boyu Wang OOD 29 3 0 31 May 2022
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations Polina Kirichenko Pavel Izmailov A. Wilson OOD 34 316 0 06 Apr 2022
OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses Robik Shrestha Kushal Kafle Christopher Kanan CML 33 13 0 05 Apr 2022
Understanding Square Loss in Training Overparametrized Neural Network Classifiers Tianyang Hu Jun Wang Wenjia Wang Zhenguo Li UQCV AAML 41 19 0 07 Dec 2021
Multi-scale Feature Learning Dynamics: Insights for Double Descent Mohammad Pezeshki Amartya Mitra Yoshua Bengio Guillaume Lajoie 61 25 0 06 Dec 2021
Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation Yeonsung Jung Hajin Shim J. Yang Eunho Yang 25 8 0 02 Dec 2021
Simple data balancing achieves competitive worst-group-accuracy Badr Youbi Idrissi Martín Arjovsky Mohammad Pezeshki David Lopez-Paz 36 173 0 27 Oct 2021
Towards Understanding the Data Dependency of Mixup-style Training Muthuraman Chidambaram Xiang Wang Yuzheng Hu Chenwei Wu Rong Ge UQCV 47 24 0 14 Oct 2021
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization Alexandre Ramé Corentin Dancette Matthieu Cord OOD 38 204 0 07 Sep 2021
Unravelling the Effect of Image Distortions for Biased Prediction of Pre-trained Face Recognition Models P. Majumdar S. Mittal Richa Singh Mayank Vatsa CVBM 43 19 0 14 Aug 2021
Fairness via Representation Neutralization Mengnan Du Subhabrata Mukherjee Guanchu Wang Ruixiang Tang Ahmed Hassan Awadallah Xia Hu 25 76 0 23 Jun 2021
OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalization Nanyang Ye Kaican Li Haoyue Bai Runpeng Yu Lanqing Hong Fengwei Zhou Zhenguo Li Jun Zhu CML OOD 40 106 0 07 Jun 2021
Quantifying and Improving Transferability in Domain Generalization Guojun Zhang Han Zhao Yaoliang Yu Pascal Poupart 40 37 0 07 Jun 2021
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization? Dinghuai Zhang Kartik Ahuja Yilun Xu Yisen Wang Aaron Courville OOD 20 95 0 05 Jun 2021
SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of Invariances in Domain Generalization Soroosh Shahtalebi Jean-Christophe Gagnon-Audet Touraj Laleh Mojtaba Faramarzi Kartik Ahuja Irina Rish 25 59 0 04 Jun 2021
Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization Damien Teney Ehsan Abbasnejad Simon Lucey Anton Van Den Hengel 23 86 0 12 May 2021
Out-of-Distribution Generalization via Risk Extrapolation (REx) David M. Krueger Ethan Caballero J. Jacobsen Amy Zhang Jonathan Binas Dinghuai Zhang Rémi Le Priol Aaron Courville OOD 215 901 0 02 Mar 2020