v1v2v3 (latest)

Fine-tuning with Very Large Dropout

1 March 2024

Papers citing "Fine-tuning with Very Large Dropout"

50 / 56 papers shown

Title
Dropout Reduces Underfitting Zhuang Liu Zhi-Qin John Xu Joseph Jin Zhiqiang Shen Trevor Darrell 120 41 0 02 Mar 2023
Learning useful representations for shifting tasks and distributions Jianyu Zhang Léon Bottou OOD 64 14 0 14 Dec 2022
SGD with Large Step Sizes Learns Sparse Features Maksym Andriushchenko Aditya Varre Loucas Pillaud-Vivien Nicolas Flammarion 129 60 0 11 Oct 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 244 138 0 19 May 2022
Universal Representations: A Unified Look at Multiple Task and Domain Learning Wei-Hong Li Xialei Liu Hakan Bilen SSL OOD 90 28 0 06 Apr 2022
Rich Feature Construction for the Optimization-Generalization Dilemma Jianyu Zhang David Lopez-Paz Léon Bottou 53 41 0 24 Mar 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 161 1,011 1 10 Mar 2022
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution Ananya Kumar Aditi Raghunathan Robbie Jones Tengyu Ma Percy Liang OODD 132 685 0 21 Feb 2022
Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning Utku Evci Vincent Dumoulin Hugo Larochelle Michael C. Mozer 113 84 0 10 Jan 2022
No One Representation to Rule Them All: Overlapping Features of Training Methods Raphael Gontijo-Lopes Yann N. Dauphin E. D. Cubuk 83 64 0 20 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 169 739 0 04 Sep 2021
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning Adrien Bardes Jean Ponce Yann LeCun SSL DML 153 945 0 11 May 2021
Universal Representation Learning from Multiple Domains for Few-shot Classification Weihong Li Xialei Liu Hakan Bilen SSL OOD VLM 93 88 0 25 Mar 2021
TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation Samuel G. Müller Frank Hutter ViT MQ 88 292 0 18 Mar 2021
Fast Adaptation with Linearized Neural Networks Wesley J. Maddox Shuai Tang Pablo G. Moreno A. Wilson Andreas C. Damianou 85 32 0 02 Mar 2021
SWAD: Domain Generalization by Seeking Flat Minima Junbum Cha Sanghyuk Chun Kyungjae Lee Han-Cheol Cho Seunghyun Park Yunsung Lee Sungrae Park MoMe 301 459 0 17 Feb 2021
Few-shot Image Classification: Just Use a Library of Pre-trained Feature Extractors and a Simple Classifier Arkabandhu Chowdhury Mingchao Jiang Swarat Chaudhuri C. Jermaine VLM 78 42 0 03 Jan 2021
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 136 268 0 18 Nov 2020
Empirical or Invariant Risk Minimization? A Sample Complexity Perspective Kartik Ahuja Jun Wang Amit Dhurandhar Karthikeyan Shanmugam Kush R. Varshney OOD 87 79 0 30 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 684 41,483 0 22 Oct 2020
In Search of Lost Domain Generalization Ishaan Gulrajani David Lopez-Paz OOD 103 1,157 0 02 Jul 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo Basel Alomair Jacob Steinhardt Justin Gilmer OOD 363 1,757 0 29 Jun 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 278 4,101 0 17 Jun 2020
Gradients as Features for Deep Representation Learning Fangzhou Mu Yingyu Liang Yin Li OOD 59 38 0 12 Apr 2020
Out-of-Distribution Generalization via Risk Extrapolation (REx) David M. Krueger Ethan Caballero J. Jacobsen Amy Zhang Jonathan Binas Dinghuai Zhang Rémi Le Priol Aaron Courville OOD 328 943 0 02 Mar 2020
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning Arsenii Ashukha Alexander Lyzhov Dmitry Molchanov Dmitry Vetrov UQCV FedML 84 320 0 15 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 390 18,897 0 13 Feb 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 163 630 0 11 Dec 2019
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization Shiori Sagawa Pang Wei Koh Tatsunori B. Hashimoto Percy Liang OOD 108 1,249 0 20 Nov 2019
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML Aniruddh Raghu M. Raghu Samy Bengio Oriol Vinyals 315 647 0 19 Sep 2019
Natural Adversarial Examples Dan Hendrycks Kevin Zhao Steven Basart Jacob Steinhardt Basel Alomair OODD 236 1,484 0 16 Jul 2019
Invariant Risk Minimization Martín Arjovsky Léon Bottou Ishaan Gulrajani David Lopez-Paz OOD 203 2,246 0 05 Jul 2019
Learning Robust Global Representations by Penalizing Local Predictive Power Haohan Wang Songwei Ge Eric Xing Zachary Chase Lipton OOD 122 967 0 29 May 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 624 4,809 0 13 May 2019
Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process Guy Blanc Neha Gupta Gregory Valiant Paul Valiant 150 147 0 19 Apr 2019
Do ImageNet Classifiers Generalize to ImageNet? Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar OOD SSeg VLM 124 1,728 0 13 Feb 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 840 0 19 Dec 2018
Moment Matching for Multi-Source Domain Adaptation Xingchao Peng Qinxun Bai Xide Xia Zijun Huang Kate Saenko Bo Wang OOD 145 1,811 0 04 Dec 2018
Domain Generalization via Conditional Invariant Representation Ya Li Biwei Huang Xinmei Tian Tongliang Liu Dacheng Tao AI4CE OOD 136 261 0 23 Jul 2018
Recognition in Terra Incognita Sara Beery Grant Van Horn Pietro Perona 102 853 0 13 Jul 2018
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 143 1,672 0 14 Mar 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 266 1,901 0 28 Dec 2017
Learning to Generalize: Meta-Learning for Domain Generalization Da Li Yongxin Yang Yi-Zhe Song Timothy M. Hospedales OOD 102 1,430 0 10 Oct 2017
Deeper, Broader and Artier Domain Generalization Da Li Yongxin Yang Yi-Zhe Song Timothy M. Hospedales OOD 127 1,453 0 09 Oct 2017
Random Erasing Data Augmentation Zhun Zhong Liang Zheng Guoliang Kang Shaozi Li Yi Yang 110 3,649 0 16 Aug 2017
Deep Hashing Network for Unsupervised Domain Adaptation Hemanth Venkateswara José Eusébio Shayok Chakraborty S. Panchanathan OOD 147 2,059 0 22 Jun 2017
Universal representations:The missing link between faces, text, planktons, and cat breeds Hakan Bilen Andrea Vedaldi 82 147 0 25 Jan 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 436 2,946 0 15 Sep 2016
Deep CORAL: Correlation Alignment for Deep Domain Adaptation Baochen Sun Kate Saenko OOD 105 3,170 0 06 Jul 2016
Deep Networks with Stochastic Depth Gao Huang Yu Sun Zhuang Liu Daniel Sedra Kilian Q. Weinberger 217 2,362 0 30 Mar 2016