Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution

21 February 2022

Papers citing "Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution"

50 / 492 papers shown

Title
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 97 20 0 23 Dec 2022
Language Models as Inductive Reasoners Zonglin Yang Li Dong Xinya Du Hao Cheng Min Zhang Xiaodong Liu Jianfeng Gao Furu Wei ReLM LRM 92 37 0 21 Dec 2022
Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization Alexandre Ramé Kartik Ahuja Jianyu Zhang Matthieu Cord Léon Bottou David Lopez-Paz MoMe OODD 124 86 0 20 Dec 2022
Foundation models in brief: A historical, socio-technical focus Johannes Schneider VLM 71 9 0 17 Dec 2022
Learning useful representations for shifting tasks and distributions Jianyu Zhang Léon Bottou OOD 76 14 0 14 Dec 2022
A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization Ashwinee Panda Xinyu Tang Saeed Mahloujifar Vikash Sehwag Prateek Mittal 126 12 0 08 Dec 2022
Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning Hongbin Liu Wenjie Qu Jinyuan Jia Neil Zhenqiang Gong SSL 66 6 0 06 Dec 2022
Fair Generative Models via Transfer Learning Christopher T. H. Teo Milad Abdollahzadeh Ngai-Man Cheung 92 25 0 02 Dec 2022
Finetune like you pretrain: Improved finetuning of zero-shot vision models Sachin Goyal Ananya Kumar Sankalp Garg Zico Kolter Aditi Raghunathan CLIP VLM 129 156 0 01 Dec 2022
Quadapter: Adapter for GPT-2 Quantization Minseop Park J. You Markus Nagel Simyung Chang MQ 64 9 0 30 Nov 2022
Transferability Estimation Based On Principal Gradient Expectation Huiyan Qi Lechao Cheng Jingjing Chen Yue Yu Xue Song Zunlei Feng Yueping Jiang 85 2 0 29 Nov 2022
Context-Aware Robust Fine-Tuning Xiaofeng Mao YueFeng Chen Xiaojun Jia Rong Zhang Hui Xue Zhao Li VLM CLIP 83 26 0 29 Nov 2022
A Time Series is Worth 64 Words: Long-term Forecasting with Transformers Yuqi Nie Nam H. Nguyen Phanwadee Sinthong Jayant Kalagnanam AIFin AI4TS 112 1,439 0 27 Nov 2022
Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization? Rishi Bommasani Kathleen A. Creel Ananya Kumar Dan Jurafsky Percy Liang 92 89 0 25 Nov 2022
Learning to Generate Image Embeddings with User-level Differential Privacy Zheng Xu Maxwell D. Collins Yuxiao Wang Liviu Panait Sewoong Oh S. Augenstein Ting Liu Florian Schroff H. B. McMahan FedML 97 31 0 20 Nov 2022
SolderNet: Towards Trustworthy Visual Inspection of Solder Joints in Electronics Manufacturing Using Explainable Artificial Intelligence Hayden Gunraj P. Guerrier Sheldon Fernandez Alexander Wong 54 9 0 18 Nov 2022
How to Fine-Tune Vision Models with SGD Ananya Kumar Ruoqi Shen Sébastien Bubeck Suriya Gunasekar VLM 127 31 0 17 Nov 2022
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization Hiroki Naganuma Kartik Ahuja S. Takagi Tetsuya Motokawa Rio Yokota Kohta Ishikawa I. Sato Ioannis Mitliagkas OOD 93 7 0 15 Nov 2022
Mechanistic Mode Connectivity Ekdeep Singh Lubana Eric J. Bigelow Robert P. Dick David M. Krueger Hidenori Tanaka 118 49 0 15 Nov 2022
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective Linyi Yang Shuibai Zhang Libo Qin Yafu Li Yidong Wang Hanmeng Liu Jindong Wang Xingxu Xie Yue Zhang ELM 183 82 0 15 Nov 2022
Using Large Pre-Trained Language Model to Assist FDA in Premarket Medical Device Zongzhe Xu LM&MA MedIm 64 0 0 03 Nov 2022
Where to start? Analyzing the potential value of intermediate models Leshem Choshen Elad Venezian Shachar Don-Yehiya Noam Slonim Yoav Katz MoMe 97 27 0 31 Oct 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 108 14 0 30 Oct 2022
Towards Generalized Few-Shot Open-Set Object Detection Binyi Su Qichuan Geng Jingzhi Li Zhongjun Zhou 117 10 0 28 Oct 2022
Learning Better Intent Representations for Financial Open Intent Classification Xianzhi Li Will Aitken Xiao-Dan Zhu Stephen W. Thomas AIFin 68 8 0 25 Oct 2022
On Fine-Tuned Deep Features for Unsupervised Domain Adaptation Qian Wang T. Breckon 55 3 0 25 Oct 2022
Amos: An Adam-style Optimizer with Adaptive Weight Decay towards Model-Oriented Scale Ran Tian Ankur P. Parikh ODL 88 6 0 21 Oct 2022
Surgical Fine-Tuning Improves Adaptation to Distribution Shifts Yoonho Lee Annie S. Chen Fahim Tajwar Ananya Kumar Huaxiu Yao Percy Liang Chelsea Finn OOD 137 214 0 20 Oct 2022
Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise Haotian Ye James Zou Linjun Zhang OOD 82 23 0 20 Oct 2022
lo-fi: distributed fine-tuning without communication Mitchell Wortsman Suchin Gururangan Shen Li Ali Farhadi Ludwig Schmidt Michael G. Rabbat Ari S. Morcos 103 24 0 19 Oct 2022
Variational Model Perturbation for Source-Free Domain Adaptation Mengmeng Jing Xiantong Zhen Jingjing Li Cees G. M. Snoek 93 27 0 19 Oct 2022
Using Language to Extend to Unseen Domains Lisa Dunlap Clara Mohri Devin Guillory Han Zhang Trevor Darrell Joseph E. Gonzalez Aditi Raghunanthan Anja Rohrbach VLM 96 35 0 18 Oct 2022
1st Place Solution in Google Universal Images Embedding Shihao Shao Qinghua Cui 3DGS 48 7 0 16 Oct 2022
Data-Limited Tissue Segmentation using Inpainting-Based Self-Supervised Learning J. Dominic Nandita Bhaskhar Arjun D Desai Andrew M Schmidt E. Rubin ... G. Gold B. Hargreaves L. Lenchik R. Boutin Akshay S. Chaudhari 59 0 0 14 Oct 2022
Are Sample-Efficient NLP Models More Robust? Nelson F. Liu Ananya Kumar Percy Liang Robin Jia VLM OOD 67 6 0 12 Oct 2022
SimSCOOD: Systematic Analysis of Out-of-Distribution Generalization in Fine-tuned Source Code Models Hossein Hajipour Ning Yu Cristian-Alexandru Staicu Mario Fritz OODD 122 5 0 10 Oct 2022
Using Interventions to Improve Out-of-Distribution Generalization of Text-Matching Recommendation Systems Parikshit Bansal Yashoteja Prabhu Emre Kıcıman Amit Sharma CML OOD 74 0 0 07 Oct 2022
Pre-trained Adversarial Perturbations Y. Ban Yinpeng Dong AAML 98 24 0 07 Oct 2022
SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data Ching-Yun Ko Pin-Yu Chen Jeet Mohapatra Payel Das Lucani E. Daniel 111 3 0 06 Oct 2022
Learning an Invertible Output Mapping Can Mitigate Simplicity Bias in Neural Networks Sravanti Addepalli Anshul Nasery R. Venkatesh Babu Praneeth Netrapalli Prateek Jain AAML 88 3 0 04 Oct 2022
Towards a Unified View on Visual Parameter-Efficient Transfer Learning Bruce X. B. Yu Jianlong Chang Lin Liu Qi Tian Changan Chen VPVLM VLM 115 36 0 03 Oct 2022
Universal Prompt Tuning for Graph Neural Networks Taoran Fang Yunchao Zhang Yang Yang Chunping Wang Lei Chen 122 58 0 30 Sep 2022
Transfer Learning with Pretrained Remote Sensing Transformers A. Fuller K. Millard J.R. Green 70 11 0 28 Sep 2022
NeurOLight: A Physics-Agnostic Neural Operator Enabling Parametric Photonic Device Simulation Jiaqi Gu Zhengqi Gao Chenghao Feng Hanqing Zhu Ray T. Chen Duane S. Boning David Z. Pan 93 19 0 19 Sep 2022
RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection Yue Song N. Sebe Wei Wang OODD 133 58 0 18 Sep 2022
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Manli Shu Weili Nie De-An Huang Zhiding Yu Tom Goldstein Anima Anandkumar Chaowei Xiao VLM VPVLM 274 309 0 15 Sep 2022
Prompt Tuning with Soft Context Sharing for Vision-Language Models Kun Ding Ying Wang Pengzhang Liu Qiang Yu Hao Zhang Shiming Xiang Chunhong Pan VPVLM VLM 77 15 0 29 Aug 2022
The Value of Out-of-Distribution Data Ashwin De Silva Rahul Ramesh Carey E. Priebe Pratik Chaudhari Joshua T. Vogelstein OODD 91 12 0 23 Aug 2022
A Multi-Head Model for Continual Learning via Out-of-Distribution Replay Gyuhak Kim Zixuan Ke Bin Liu VLM CLL OODD 73 31 0 20 Aug 2022
Intersection of Parallels as an Early Stopping Criterion Ali Vardasbi Maarten de Rijke Mostafa Dehghani MoMe 67 5 0 19 Aug 2022